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ABSTRACT 


Iterative  protocols  for  adaptive  routing  in  line  and  message 

switched  data  communication  networks  are  presented  in  this  thesis. 

The  protocols  have  the  following  features: 

% 

1.  Distributed  computation  is  used  in  the  sense  that  each  node  in 
the  network  bases  all  its  decisions  on  control  messages  received 
only  from  its  neighbors.  Thus,  each  node  in  the  network  determines 
individually  onto  which  of  its  outgoing  links  to  send  the  flow, 
addressed  to  a specific  destination.  The  control  messages  exchanged 
between  neighbors  contain  information  about  network  connectivity, 
network  congestion  and  link  failures. 

2.  Loop-free  routing  for  each  destination  is  maintained  in  the  network 
at  all  times.  Generally,  prevention  of  loops  results  in  saving 
resources  and  reduction  in  delay.  In  addition,  loop-free  routing 
establishes  a partial  ordering  on  the  set  of  nodes  of  the  network. 

The  latter  property  is  extensively  utilized  throughout  this  work. 

3.  Failsafe  and  deadlock-free  operation  of  the  protocols  is 
guaranteed,  meaning  that  after  arbitrary  failures  and  additions  of 
links  and  nodes,  the  network  recovers  in  finite  time.  Recovery 
means  that  routing  paths  are  provided  between  all  connected  nodes. 

4.  For  stationary  input  traffic  statistics  and  fixed  topology  the  protocols 
are  optimal.  They  reduce  network  delay  at  each  iteration  and  minimum 
average  delay  over  all  routing  assignments  is  obtained  in  steady-state. 
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Proofs  of  all  features  are  provided. 

The  protocols  are 
input  requirements  are 
links  or  nodes  fail  or 


intended  for  quasi-static  applications  where  tht 
slowly  changing  with  time  and  where  occasionally 
are  added  to  the  network. 
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Proofs  of  all  features  are  provided. 

The  protocols  are  intended  for  quasi-static  applications  where  the 
input  requirements  are  slowly  changing  with  time  and  where  occasionally 
links  or  nodes  fail  or  are  added  to  the  network. 
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NOTATION 
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^(j) 


fik^ 


V” 

^ik0) 


D 
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d.(j) 

SINK 

d. 

1 

b. 
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GLOSSARY  OF  NOTATIONS 

DEFINITION 

- number  of  nodes  in  a network. 

- set  of  links  in  a network. 

- directed  link  from  node  i to  node  k. 

- average  traffic  entering  the  network  at  node  i 
and  destined  for  node  j . 

- average  flow  in  link  (i.k)  of  traffic  destined 
for  node  j . 

- total  average  traffic  in  link  (i.k)  . 

- the  set  of  link  flows . 

- total  average  traffic  at  node  i destined  for  node  j. 

- fraction  of  the  node  flow  t^(j)  that  is  routed  through 
link  (i.k)  . 

- capacity  of  link  (i.k). 

- average  delay  per  unit  time  of  all  traffic  sent  over 
link  (i  ,k)  . 

- total  delay  in  the  network  per  time  unit. 

- average  unit  of  traffic  length. 

- propagation  delay  in  link  (i.k). 

- nodal  processing  time  in  node  k. 

- estimated  marginal  delay  of  node  i from  destination  j . 

- destination  node. 

- estimated  marginal  delay  of  node  i from  SINK. 

- blocking  status  of  node  i. 

- current  counter  number  of  node  i . 
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GLOSSARY  OF  NOTATIONS  (Cont.) 


NOTATION 

DEFINITION 

MSG(m,d,b , £) 

- 

updating  message  sent  by  node  l. 

FAIL(i) 

- 

failure  detected  on  link  (i,5.). 

WAKE ( £) 

- 

link  (i.JI)  becomes  operational. 

Fi(l) 

- 

status  of  link  (i,Jl)  as  seen  from  node  i. 

NjCi) 

- 

the  number  m received  from  neighbor  l during  the 

current  iteration. 

D'i* 

- 

estimated  (or  calculated)  marginal  delay  on  link  (i,J 

Di(^) 

- 

sum  of  D'  and  last  number  d received  at  i from 

lJc 

neighbor  l. 

B±C^) 

- 

blocking  status  of  neighbor  l as  known  at  i. 

zA(t) 

— 

a synchronization  number  indicating  the  iteration 

upon  which  the  link  (i,£)  can  be  brought  up. 

RjCi) 

- 

status  of  neighbor  l (being  a son) . 

mx. 

1 

— 

the  largest  number  m received  by  i up  to  the 

current  time  from  all  neighbors. 

Pi 

- 

preferred  son  of  node  i . 

CT 

a flag  indicating  the  number  of  transitions  the 

Finite-State-Machine  has  already  performed  triggered 

by  the  current  message. 

REQ(m) 

— 

request  message  destined  for  SINK  to  start  an  iterat 

with  counter  number  (m+1) . 

C.f  A. 

1 l 

- 

sets  of  neighbors  of  node  i. 

SONi 

- 

set  of  all  sons  of  node  i. 

n 

- 

a parameter . 

GLOSSARY  OF  NOTATIONS  (Cont.) 


NOTATION 


DEFINITION 


PC(m) 

RG 

SI,  S2,  S2.S3 
Cl,  C2 

T12.T13.T21,  'l 
T22,  T23.T22,  j= 
T32.T22.T23  J 
S; 


instant  of  occurence  of  proper  completion  of  an 
iteration  with  counter  number  m. 
routing  graph. 

states  of  the  Finite-State-Machine. 

changes  performed  in  the  Finite-State-  Machine. 

transitions  performed  in  the  Finite-State-Machine 

state  of  node  i. 
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CHAPTER  1 
INTRODUCTION 

Many  efforts  have  been  and  are  devoted  to  the  design  and  the 
analysis  of  data  communication  networks,  which  provide  the  facility 
of  interconnection  between  a number  of  users  for  sharing  resources 
between  them.  This  kind  of  networks  includes  time  shared  computer 
systems,  medical  data  networks,  bank  transaction  systems,  airline 
reservation  systems,  multipurpose  data  networks,  (e.g.  AT&T, 

Western  Union},  large  scale  computer  networks  (e-g-  the  ARPA  network 
Advanced  Research  Projects  Agency},  etc.  [SCHW  72a]. 

Generally  speaking,  a data  network  consists  of  a set  of  users 
[.computers,  terminals,  displays,  etc  .} connected,  by  a communication 
subnetwork  that  is  in  charge  with  transferring  data  between  the  users. 

In  this  work  we  will  be  concerned  with  the  communication  subnet.  The 
latter  consists  of  nodes  which  exchange  data  with  each  other  through 
a set  of  connecting  links . The  nodes  CJMP-in  ARPA}  are  real-time 
computers,  with  limited  storage  and  processing  resources,  which  perform 
some  basic  functions,  the  main  of  which  being  to  direct  the  data  that 
passes  through  them.  The  connecting  links  are  some  type  of  communication 
channels  of  relatively  high  bandwidth  and  reasonably  low  error  rate. 

The  subnet  topology  design  is  usually  one  of  the  difficult  problems  in 
the  design  phase.  However,  for  the  purposes  of  this  work,  we  do  not 
consider  this  problem  . and  assume  a general,  geographically  distributed 
topology,  in  which  each  node  can  have  multiple  paths  to  other  nodes. 


Clearly,  there  must  exist  some  set  of  disciplines  governing  the 
flow  of  data  between  the  users,  between  the  users  and  the  nodes,  and 
between  the  nodes  themselves.  In  this  work  we  are  only  concerned  with 
the  rules  used  by  the  nodes  to  determine  in  which  directions  to  deliver 
the  data  traffic,  from  the  source  node  to  the  destination  node,  namely 
with  the  routing  policy. 

The  complication  of  the  routing  problem  in  a network  is  commonly 
a question  of  assumptions,  formulations  and  goals,  involved  in  it.  . 
The  more  assumptions  we  make,  it  is  expected  the  less  complicated  the 
problem  will  be.  However,  the  designer  and  the  analyst  certainly  wish 
to  make  as  few  assumptions  as  possible.  The  formulation  is  probably 
a matter  of  convenience,  and  the  goals  can  differ  in  various  problems. 

In  the  next  subsections  we  describe  some  network  types,  routing 
policies  and  control  schemes  that  are  commonly  used  in  data  networks. 
Also  an  outline  of  the  following  chapters  is  given  and  the  contribution 
of  this  work  is  emphasized. 

1.1:  Network  Types 

Corresponding  to  any  routing  policy,  two  hasic  types  of  networks 
are  in  use  or  in  development  - the  line  or  circuit  switching 
type,  and  the  message  or  packet  switched  type.  These  two  types  are 
distinct  techniques  for  communication  among  the  nodes  of  the  subnet, 
and  any  combination  of  the  two  is  possible. 
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1.1.1:  Line  Switched  Networks 


In  a line  switched  network  [TYM  71] , which  is  very  similar  to  the 
telephone  network,  the  source  and  the  destination  nodes  are  connected 
by  one  or  more  communication  paths  that  are  established  at  the 
beginning  of  the  connection,  and  are  cancelled  when  the  desired 
connection  is  terminated  or  when  the  path  is  disrupted  by  failures. 

In  other  words,  the  connecting  paths  between  the  source  and  the  destination 
are  dedicated  before  any  data  messages  are  transmitted  from  the  source 
to  the  destination  through  the  selected  path  and  exist  for  the  duration 
of  the  connection.  In  different  routing  strategies  these  paths  may  either 
remain  fixed  until  the  connection  is  over,  or  be  changed,  but  not  cancelled, 
during  the  existence  of  the  connection.  One  version  of  the  line-switching 
strategy  is  "virtual  line-switching"  [TYM  71] , where  data  is  forwarded 
according  to  the  established  paths  connecting  the  source  and  the  destination, 
but  messages  corresponding  to  different  connections  are  multiplexed  together 
on  each  link  .and  demultiplexed  on  the  other  end  of  the  link.  In  this  way, 
the  portion  of  the  link  capacity  used  by  each  call  is  varying  according 
to  its  momentary  transmission  requirement. 


1.1.2:  Message  Switched  Networks 

In  a message  switched  network  [MCQ  77] , each  message  makes  its  own 
way  to  the  destination,  and  usually  messages  corresponding  to  the  same 
destination  will  travel  on  different  paths  which  are  not  predetermined. 
In  this  type  of  network,  a message  entering  it,  is  first  stored  m the 
source  node  until  its  time  comes  to  be  sent  on  an  outgoing  link  to  a 
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neighboring  node.  CJhe  selection  of  the  neighbor  is  exactly  the  routing 
policy).  Having  been  received  by  that  node,  it  is  stored  again  in  a 
queue  until  it  is  being  sent  forward  to  the  next  node.  Thus  the  message 
continues  to  pass  links,  and  be  queued  at  nodes  until  it  reaches  its 
destination. 

Packet  switched  network  is  fundamentally  the  same  as  message  switched 
network,  except  that  messages  are  split  into  a number  of  small  segments 
of  maximum  length  called  packets. 

1.2:  Routing  Policies  Classification 

Several  classification  schemes  have  been  proposed  to  characterize 
routing  policies.  The  scheme  we  use  is  according  to  how  dynamic  the 
policies  are.  On  one  end  of  the  scale  we  have  the  purely  static 
strategies,  and  on  the  other  end  we  have  the  completely  dynamic  ones. 
Quasi-static  strategies  lie  in  between. 

1.2.1:  Static  Routing 

In  the  purely  static  or  deterministic  situation,  the  set  of  rules 
dictating  the  fractions  of  traffic  with  a given  destination  sent  by  a 
node  to  each  of  its  outgoing  links,  is  fixed.  These  fractions  are  decided 
upon,  under  several  criteria,  before  the  establishment  of  the  network,  by 
making  various  assumptions  about  the  node  and  link  locations,  and  the 
capacities  of  the  links.  [FRAN  71,  CHO  72,  FRA  73,  GER  73,  CAN  74]. 

The  decisions  are  fixed  in  time,  and  do  not  change.  Static  routing 
strategies  are  non-adaptive  in  nature  and  lack  the  ability  to  cope  with 
changing  network  conditions  such  as  failures  of  nodes  and  links  and/or 
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changes  in  traffic  requirements,  and  as  such  are  too  unreliable  and 
inefficient  to  be  considered  in  practice  for  nontrivial  size  networks. 
However,  their  simplicity  makes  them  very  attractive  to  use  at  the  design 
phase  of  the  network. 


1.2.2:  Routing 

Completely  dynamic  routing  strategies  allow  continuous  changing 
of  routes  as  a function  of  time,  as  well  as  a function  of  the  network 
states  such  as  traffic  requirements,  queue  lengths  and  component  failures. 
They  are  thus  supposed  to  be  able  to  adapt  to  changing  conditions  in  the 
network.  Dynamic  routing  is  much  more  advantageous  since  it  is  adaptive. 
However,  it  has  some  inherent  drawbacks,  the  main  of  which  being  that 
it  requires  large  amounts  of  overhead  per  message  for  purposes  of  adressing, 
reordering  at  destinations  etc. 


1.2.3:  Quasi-Static  Routing 

Given  the  advantages  and  the  drawbacks  of  each  of  the  two  already 
described  policies,  naturally,  one  should  try  to  devise  policies  that  can 
possibly  acquire  some  of  the  advantages  of  both.  Using  a quasi-static 
routing  strategy  is  one  possiblity,  since  it  is  adaptive  in  nature, 
but  the  routes  can  not  be  continuously  changed  [GALL  77,SEG  77a, 

SEG  77b].  In  this  strategy,  changes  of  routes  are  allowed  only  at  given 
intervals  of  time,  and/or  whenever  a need  to  do  so  arises  because 
extreme  situations  occur  in  the  network,  such  as  link  and  node  failures 
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or  recoveries. The  time  intervals  between  routing  changes  should  be 
relatively  long,  so  that  most  of  the  time  messages  are  sent  in  order, 
causing  a serious  reduction  in  the  overhead  needed,  but  they  should  not 
be  too  long,  otherwise,  the  inferior  of  the  fixed  routing  would  be 
revealed. 

I 

In  order  to  allow  adaptivity,  the  quasi-static  routing  procedure 
has  to  sense  changes  in  the  network  status  and  in  traffic  requirements, 
and  then  to  route  messages  accordingly,  for  example,  congested  or  damaged 
portions  of  the  network  should  be  avoided.  Adaptivity  to  failures 
is  of  great  importance  in  order  to  maintain  a good  grade  of  service  for 
the  network. 


1.3:  Routing  Information 

Generally,  adaptive  routing  strategies  base  their  decisions  on 

measured  values  which  describe  the  salient  features  of  the  network.  In 

l 

completely  dynamic  strategies  the  values  are  measured  continuously,  and 
actually  are  the  instantaneous  states  of  the  queues  at  the  outgoing  links 
of  the  nodes.  In  quasi-static  strategies  the  varying  values  are 
periodically  measured,  and  consist  of  quantities  such  as  the  queues 
at  the  links,  traffic,  or  the  status  of  the  network.  These  measurements 
are  reffered  to  as  routing  information. 
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1.4:  Networks  Control 

Gathering  the  routing  information,  two  main  approaches  exist  to 
conduct  the  routing  procedures  - the  centralized  control  scheme  and 
the  decentralized  one,  which  is  also  known  as  distributed  control. 


1.4.1.:  Centralized  Control 

In  a centralized  adaptive  policy,  the  nodes  collect  the  necessary 
routing  information  for  making  the  routing  decisions,  and  send  it  to 
a special  node  in  the  network,  which  is  the  central  node  or  the  governor. 
Receiving  the  information,  the  central  node  has  a global  status  picture 
of  the  network,  and  can  dictate  its  routing  decisions  back  to  the  nodes 
for  actual  use  [TYM  71,  BRO  75].  The  decisions  are  naturally  based 
upon  some  criteria,  in  order  to  optimize  the  routing  in  the  network 
in  some  sense. 

The  centralized  policy  seems  simple  and  straightforward,  and  has 
some  advantages,  mainly  due  to  the  availability  of  global  status 
information  at  one  place  in  the  network.  Since  the  computations  needed 
to  make  the  decisions  are  conducted  only  by  the  central  node,  the  used 
algorithms  might  be  very  sophisticated,  and  at  the  same  time  simple 
to  understand.  It  is  possible  to  achieve  several  goals  such  as  "optimal" 
routing,  avoiding  "loops"  etc.  Also  the  nodes  in  the  network  are  relieved 
of  the  troublesome  task  to  make  routing  decisions,  so  overhead  is  saved 
at  each  node. 
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In  practice,  however,  the  centralized  control  scheme  has  several 
drawbacks  and  inherent  weaknesses.  Should  the  central  node,  or  the  links 
connecting  it  to  the  network,  fail,  or  should  some  part  of  the  network 
become  isolated,  then  all  or  some-  of  the  network  nodes  remain  without 
routing  decisions  for  actual  use,  and  a part  or  the  entire  network  cannot 
operate  anymore. 

Another  possible  difficulty  may  arise  when  nodes  or  links  fail.  The 
central  node  must  be  notified  of  such  failures.  However,  the  failed 
components  might  lie  on  the  paths,  previously  determined  by  the  governor, 
between  the  nodes  trying  to  report  the  failure  and  the  central  node. 

We  also  notice  that  since  the  central  node  conducts  all  the  computations, 
it  is  likely  to  be  very  heavily  loaded. 

Finally,  the  unbalanced  demands  on  network  link  bandwidth,  is  a clear 
drawback.  Since  routing  information  and  decisions  go  to  and  from  the 
central  node  through  its  outgoing  links,  these  links  are  heavily  utilized, 
when  at  the  same  time  other  links  in  the  network  might  be  bored. 

Apparently,  this  may  also  limit  the  size  of  the  network. 

Of  course,  the  simple  centralized  control  scheme  might  be  improved 
in  such  ways  that  some  of  its  weaknesses  will  be  overcome.  For  instance, 
the  governor  may  have  back-up  centers,  on  stand-by,  ready  to  take  the  control 
of  the  network,  whenever  it  fails.  Eventually,  there  arises  the  problem  of 
identifying  which  node  is  in  control  of  which  nodes.  Such  identification 
is  essential  for  proper  work  of  any  centralized  control. 

A natural  way  to  overcome  the  fundamental  weaknesses  of  the  centralized 


control  scheme  is  to  wonder  why  shouldn't  all  the  nodes  in  the  network  be 
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"centers"  and  participate  in  the  routing  decisions.  This  leads  us  directly 
to  the  distributed  control  scheme. 


1.4.2:  Distributed  Control 

Distributed  adaptive  control  schemes  have  neither  the  inherent 
inefficiency  and  unreliability  of  fixed  routing,  nor  the  unreliability 
and  size  limitations  of  centralized  control  schemes.  Here  each  node 
needs  to  individually  perform  the  necessary  computations,  and  to  make 
the  routing  decisions  in  collaboration  with  its  adjacent  nodes  called 
neighbors.  [STE  77,  GALL  77,  SEG  77a,  MCQ  77].  It  is  usually  done  by 
storing  the  routing  information  in  routing  tables  at  each  node,  and  using 
the  tables  to  identify  the  output  link  each  message  has  to  select,  for 
each  destination.  The  tables  might  be  updated  periodically  or  only  when 
it  matters  (^synchronously]  or  any  combination  of  both,  by  using  the 
routing  information  each  node  collects  internally  and  receives  from 
its  neighboring  nodes. 

In  most  commonly  used  distributed  adaptive  schemes,  each  node 
estimates,  by  a certain  procedure,  the  "distance"  it  expects  a message 
would  have  to  travel  in  order  to  reach  each  possible  destination,  if 
the  message  is  transmitted  over  each  of  the  outgoing  links,  and  stores 
these  estimates  in  the  routing  table.  The  "distance"  is  a measure,  which 
numerically  expresses  the  quantity  that  the  routing  procedure  is  to  minimize 
in  order  to  achieve  the  desired  performance  of  the  network,  as  defined  at 
the  design  stage. 
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Each  node  in  a network  of  N nodes  has  a routing  table  which  is 
typically  composed  of  N-l  entries,  one  for  each  destination.  Each  entry 
indicates  the  estimated  minimal  distance  from  this  node  to  each  destination 
and  also  the  next  node  the  message  must  pass  on  its  way  to  the  destination, 
along  the  minimal  distance  path. 

The  routing  table  of  each  node  is  updated  as  follows.  Each  node  selects 
the  minimal  estimated  distance  for  each  destination  and  sends  these  estimates 
to  each  of  its  neighboring  nodes.  Receiving  these  estimates,  each  node 
constructs  its  own  routing  table  by  adding  its  neighbors'  received 
estimates,  to  its  own  estimates  of  distance  to  each,  of  its  neighbors.  For 
each  destination,  the  routing  table  is  then  constructed  to  indicate  the 
selected  outgoing  link,  for  which  the  sum  of  the  estimated  distance  to 
the  neighbor  and  the  estimated  distance  from  the  neighbor  to  the  destination, 
is  minimal. 

Distributed  routing  schemes  are  not  lacked  of  weaknesses.  Since  there 
is  no  place  in  the  network  where  global  status  of  the  network  and  its 
topology  are  available,  then  temporary  "loops"  may  exist  within  the  net- 
work, and  also  it  becomes  twofold  harder  to  maintain  failsafe  operation 
of  the  network. 


1.5:  Routing  Performance  Evaluation 

Any  specific  routing  assignment  algorithm  is  to  achieve  certain 
goals,  and  to  fulfill  some  criteria.  It  is  to  be  simple,  to  adapt  to 
changes,  tc  converge  to  an  accurate  and  stable  routing  assignment  under 
stationary  conditions,  and  it  is  to  optimize  some  cost  functions. 


k 
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The  cast  function  most  commonly  used  to  evaluate  the  performance  of  a 
routing  algorithm  is  the  delay  experienced  by  messages  when  traveling 
through  the  network.  The  delay  is  composed  of  propagation  delays, 
transmission  delays,  nodal  processing  delays  and  queueing  delays 
Clearly,  the  delay  must  be  minimized  for  good  grade  of  performance 
of  the  algorithm. 

Other  criteria  may  be  reliability,  throughput,  which  are  to  be 
maximized,  and  network  cost  which  is  to  be  minimizec. 

A number  of  algorithms  have  been  proposed  to  achieve  some  of  these 
goals  for  static  routing  [FRA  73,  GER  73,  CAN  74],  as  well  as  for 
centralized  adaptive  routing  [BRO  75],  and  distributed  adaptive 
routing  [STE  77,  GALL  77,  SEG  77a,  NAYL  77],  Some  of  these  algorithms 
will  be  discussed  presently. 


1.6:  Contribution  of  this  Thesis 

In  this  thesis  we  develop  distributed  routing  protocols  which  are 
natural  extensions  of  three  known  protocols  introduced  in  recent  papers 
[GALL  77],  [SEG  77a]  and  [SEG  77c].  In  [SEG  77c]  a failsafe  distributed 
protocol  which  maintains  a single  optimal  route  from  each  node  to  the 
destination  is  developed  In  [GALL  77]  and  [SEG  77a]  quasi-static 
distributed  routing  protocols  which  minimize  the  total  expected  delay 
in  a network  with  fixed  topology,  are  proposed.  The  features  of  the  above 
protocols  are  unified  in  our  thesis,  namely  our  protocols  are  both 
distributed  and  failsafe  and  in  addition  they  indicate  the  exact  amounts 


of  flow  splitting,  so  that  minimum  average  delay  is  obtained  in  the 
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network  in  steady-state. 


1.7:  Outline  of  Chapters 

In  Chapter  2 we  present  the  protocols  of  [GALL  77]  and  [SEG  77a] 
that  provide  the  basis  of  the  present  work.  A different  presentation  from 
that  in  the  references  is  used  in  order  to  facilitate  the  explanation  of 
our  failsafe  distributed  protocols  described  in  Chapters  3 and  4.  In 
Chapter  3 our  protocols  are  described  in  detail,  their  properties  are  stated 
and  proofs  for  these  properties  are  given  in  Appendix  A and  Appendix  B. 

The  protocol  is  completed  in  Chapter  4 and  Chapter  5 deals  with  a 
simulation  program,  given  in  Appendix  C,  which  was  developed  to  check 
the  protocol  of  individual  nodes.  Conclusions  are  discussed  in  the  final 
Chapter. 
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CHAPTER  2 


ROUTING  PROTOCOLS  FOR  FIXED  TOPOLOGY 


2.1:  Introduction 

In  this  chapter  the  routing  protocols  proposed  by  R.G.  Gallager 
in  [GALL  77]  and  by  A.  Segall  in  [SEG  77a]  are  described.  First,  two 
network  models  are  presented  and  some  definitions  and  equations  are  stated. 
Then  the  protocols  are  described  by  using  a different  presentation  than 
in  the  above  references. 


2.2:  The  Models 

2.2.1:  General  Model 


Consider  a communication  network 
the  integers  (1,2,3,. . .,N),  and  a set 
link  from  node  i to  node  k be  denoted 
in  Fig.  1. 

Let  us  now  define  some  symbols: 


consisting  of  N nodes  denoted  by 
L of  directed  links.  Let  a 
by  (ilk).  An  example  is  given 


£ikCj) 


f. 


ik 


Vj) 


average  traffic  entering  the  network  at  node  i and  destined 
for  node  j . 

average  flow  in  link  Ci.k)  of  traffic  destined  for  node  j. 
total  average  traffic  in  link  (i>k),  f^k  ■ [ f^Cj)- 
total  average  traffic  at  node  i destined  for  node  j . 
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■ fraction  of  the  node  flow  t^(j)  that  is  routed  through  link 
ti.k). 

Cik  = capacity  of  link  (_i,k). 

For  later  purposes,  we  shall  use  a special  notation  to  indicate 
that  node  k is  a neighbor  of  i,  namely  that  (i,k)  e L . The  notation 
will  be  F^Ck)  = UP.  The  reason  for  using  this  notation  will  become 

apparent  when  dealing  with  topological  changes  in  Chap.  3. 

• 

It  is  now  possible  to  express  the  law  of  conservation  of  flow  at 
each  node  by  various  equations.  Different  equations  are  used  for  line 
switched  and  message  switched  networks,  the  reason  being  that  the  controlled 
quantities  are  the  flows  of  data  for  the  former  and  the  fractions  of  the 
flow  for  the  latter. 


2.2.2:  Line  Switched  Network 

The  flows  f^(j)  must  satisfy: 

I f kCj)  ■ I “ r.  (jl  for  all  i.j.i^j  (2-1) 

k:F.(k)«UP  KF^l-UP  X 

if  j 


£lkU3  * 0 for  all  i,j,k,  i^j  (2.2) 

fLk  - ! fikCj)  < Cik  for  all  (i,k)  z L (2.3) 

J 
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2.2.3:  Message  Switched  Network 


The  fractions  4>ik(j)  roust  satisfy: 


tiCjl  - l 'iQI+mCU  - r.Ql 
2:FiQll-UP  X 

for  all  i,  j , i»<j 


<>ikU2  ^ 0 ; l 4>.v(j}  - 1 


k:F.Qc)*UP 


ik' 


for  all  i,j ,k,  ijj 


£ik  “ l tiCj)$ikCj)  < Cik  for  a11  U>kl  e L 


(2.5) 


(2.6) 


In  [GALL  77]  it  is  proved  that  if  for  each  i,j,(ij*j)  there  is  a 
routing  path  from  i to  j which  means  there  is  a sequence  of  nodes 
i,k,4, . . . ,m,  j such  that  > 0,  $kJl  > 0,  ....  $mj(j)  > 0 then  the  set 
of  equations  (2.4)  has  a unique  non-negative  solution  for  t^(j), 
i =>  1,2, ...  ,N. 


2.3:  Delay 

Let  Dik  be  the  average  delay  per  time  unit  (seconds),  of  all 
traffic  sent  over. link  (i,k).  Explicitly,  D^k  is  the  average  delay  per 
unit  of  traffic  (bit,  message,  packet)  multiplied  by  the  amount  of  traffic 
per  time  unit  passing  through  link  (i.k  ).  We  shall  assume  that  Dik  is 


i 
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only  a function  of  the  total  flow  of  traffic  f^  transmitted  through 
link  (i,k).  Some  of  the  consequences  of  this  assumption  are  given  in 
[GALL  77]. 

The  objective  of  the  algorithms  presented  here  is  to  minimize  the 
average  delay  per  unit  of  traffic.  However,  since  the  total  arrival  rate 
into  the  network  is  independent  of  the  routing  policy,  this  objective 
might  be  achieved  by  minimizing  the  total  delay  in  the  network  per  time 
unit,  which  is  given  by: 


l 

Ci.kleL 


Dik^ikl 


(2.7] 


The  quasi-static  algorithms  presently  described  perform  this  minimization 
by  iteratively  changing  the  routing  assignments,  while  keeping  the  flow 

feasible  at  each  iteration. 


It  should  now  be  pointed  out  that  the  algorithms  do  not  require  any 
explicit  knowledge  of  the  functions  D^C')  In  [KLEI  64]  it  is  shown  that 
under  several  assumptions  the  delay  in  steady  state  takes  the  explicit 

term 


D 


ik 


(2.8a) 


Another  more  general  well-known  form,  where  propagation  and  nodal  processing 
times  are  taken  into  account,  is  given  by  [GERLA  77]: 


°ik 


fik[  C 


ik 


ik 


u(Pik  * 


KikU 


(2.8b) 


wnere 
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■jj  is  the  average  unit  of  traffic  length;  * propagation  delay  in 
link  (i,k);  = nodal  processing  time  in  node  k. 

For  the  purposes  of  this  work,  it  is  enough  to  assume  only  the 
following  reasonable  properties  of  the  functions 

is  a non-negative  continuous  increasing  function  of  f^»  with 
continuous  first  and  second  derivatives.  (2.9a) 


is  convex  U 


(2.9b) 


lim 

futile 


Dik«ik> 


dD.. 

D,ikCfik>  ” 0 *for  aU  fik’  whexe  D'iktfik5  m d 


(2.9c) 

(2.9d) 


Observe  that  the  functions  in  2.8  indeed  have  these  properties. 


2.4:  Necessary  and  Sufficient  Conditions  for  Minimum  Delay 

In  [GALL  77]  and  [SEG  77a]  necessary  and  sufficient  conditions  for 
minimum  delay,  have  been  derived  for  mess age -switched  and  line-switched 
networks,  respectively.  Here  we  only  indicate  their  results. 
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2.4.1:  _Message_Switched  Network 


If  for  each  (i,k)  el,  the  functions  DikC#ikl  have  the  properties 
given  in  (2. 9],  then  a necessary  condition  for  ♦“($i]c(j))  to  minimize 
Dt  over  the  set  of  <J>  satisfying  C.2.6]  is  that  there  exists  a set  of 
numbers  such  that 


3DT 


• x.cj], . *ikcn  > o 
I * ^(j).  *ikU)  - o 


C2.10) 


A sufficient  condition  to  minimize  DT  is: 


D'.v(f.J  ♦ 


3Dt  3Dt 


ik^ikJ  3rk(j)  ' 3r.(j) 

for  all  i,j,k,  i^j , Ci.kjci 

The  last  expression  C2.ll)  has  been  shown  to  be  equivalent  to 
3DX  3Dt 

* SrjJtjT  ■ ‘“W  * sTcJT  1 i0 

for  all  i,j,k,  ij<j , (i,k)e£ 

with  equality  for  these  i,k,j,  such  that  <frikCi)  is  strictly  positive. 


(2.11) 


(2.12) 


The  quantity  3D^./3r^(j)  is  the  incremental  delay  caused  by  a small 
increment  in  the  input  ^(j)  and  might  be  calculated  as  follows: 


l ♦ikCj)lD'ikCfik3  + wm ] 

k:F.(k)-UP  1K  1 rkUJ 


(2.13) 


2.4.2:  Line  Switched_Network 

Assume  that  the  set  of  flows  satisfying  (2.1),  (2.2),  (2.3)  is 
nonempty,  and  let  the  functions  have  the  properties  given  in 

C2.9'1  for  each  (i,k)eL  . Then  DT  is  minimized  by  the  flows 
f • { f ±k (j ) ) if  and  only  if  there  exists  a set  of  numbers  X-iX.^)} 
such  that 


D’iktfik>  * Xk^ 


- X.(j)  if  fik(j)  > 0 


* ^(j)  if  fi]c(j)  “ 0 


(2.14) 


for  all  i,j,k,  ii*  j , (i.k)e  L , where  X^(j)  ■ 0.  Observe  that  if  the 
input  flow  r^Cj)  is  increased  by  an  incremental  quantity  <S r ^ ( j ) and 
everything  else  is  held  fixed,  then  the  minimum  delay  will  be  increased 
by  the  incremental  quantity  X^ (j) • 6^ (j ) . Therefore  the  coefficients 
t A ^ (j ) > might  be  intepretated  as  marginal  delays. 

To  have  a common  notation  for  line-switched  and  message  switched 
networks,  it  will  be  convenient  to  denote  both  3D,j./3r^(j)  in  (2.13), 

(2.14)  and  ^(j)  in  (2.15)  by  a common  notation.  Therefore  we  shall  use 
d^j)  to  denote  both  the  quantity  30^/3^  (j)  in  the  message- switched  model 
and  Xi(j)  in  the  line-switched  model.  Observe  that  di(j)  is  the  marginal 
node  delay,  while  D'ik^ik^  t*'e  marginal  link  delay. 


In  this  section  the  routing  protocols  converging  to  the  minimum  delay 
are  first  briefly  discussed.  Then  a formal  presentation  of  the  protocols 
is  given,  which  is  somewhat  different  from  that  in  [GALL  77]  and  [SEG  77a]. 
Here,  the  operations  required  by  the  algorithms  at  each  node  are  summarized 
as  a Finite-State-Machine  with  transitions  between  states  triggered  by  the 
arrival  of  control  messages.  Control  messages  are  sent  between  neighbors, 
queued  at  the  receiving  node  and  processed  on  a first-come-first- served 
CFIFO)  basis.  The  processing  of  a control  message  consists  of  temporarily 
storing  it  in  suitable  memory  locations,  followed  by  activation  of  the 
Finite-State-Machine,  which  takes  the  necessary  actions  and  performs  the 
appropriate  state  transitions.  Some  variables  are  used  as  conditions  for 
the  execution  of  transitions,  and  their  values  might  be  changed  by  the 
transitions. 

For  readers  familiar  with  Gallager's  algorithm,  we  note  that  we 
introduce  here  a slight  modification.  In  [GALL  77] , the  updating  of  the 
quantities  di(j]-3D^,/3r^(j]  is  performed  while  the  protocol  propagates 
from  each  destination  upstrsam  in  the  network,  while  the  timing  of  the 
actual  rerouting  is  left  arbitrary.  For  later  purposes,  related  to 
topological  changes,  it  will  be  convenient  that  we  introduce  already  at 
this  point  a certain  sequencing  for  rerouting.  Specifically,  the  update 
of  (di(j]}  will  be  performed  as  before  while  the  protocol  propagate* upstream . 
but  now  we  will  also  have  a propagation  of  the  protocol  in  the  downstream 
direction,  during  which  the  nodes  will  actually  change  their  routing. 
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2.5.2:  Informal  Descr igt i?n_o£_the_Protocol 

Considering  the  optimality  conditions  for  the  two  models,  the  general 
structure  of  the  algorithms  should  be  clear.  A node  i will  have  to  increase 
traffic  destined  for  node  j on  links  Ci.k)  with  small  marginal  delay 
* d^Cj)  anc*  t0  ^ecrease  traffic  on  those  with  large  marginal 

delay. 


Obviously,  in  addition  to  the  quantities  d^Cj)  that  it  receives  from 
neighbors,  each  node  i will  need  the  marginal  delay  over  each 

of  its  outgoing  links.  can  be  obtained  by  node  i by  estimating 

f^k  and  using  appropriate  formulas  for  D'^Cf^)-  However,  each  formula 
involves  many  assumptions,  so  node  i should  preferably  estimate 
D'ik(fik)  directly.  Such  estimation  procedures  have  been  developed  in 
[SEG  77b],  and  from  now  on  we  assume  that  each  node  i continuously  estimates 
or  calculates  the  marginal  delay  D'^Cf^)  over  each  of  its  outgoing  links 
Ci.k) . 


We  should  also  note  here  that  the  optimality  conditions  clearly 
show  that  different  destinations  are  not  related,  so  that  the  protocols 
may  evolve  independently  from  one  destination  to  another.  That  is  why 
the  protocols  are  presented  for  a given  fixed  destination  j which  is  denoted 
by  SINK  from  now  on. 


Before  proceeding, the  following  definitions  are  needed. 
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Son;  For  message  switched  network:  All  neighbors  k of  node  i (namely 

all  nodes  k s.t.  F ^ Ck with  ^^CSINK)  > 0 are  called  its  sons 
(see  Fig.  2] . 

For  line  switched  network:  All  neighbors  k of  node  i (namely 
ail  nodes  k s.t.  F^k)  »UP)  with  f^CSINK]  > 0 are  called  its  sons. 

In  case  that  f^(SINK)  ■ 0 for  all  neighbors  k,  then  node  i has 
exactly  one  son;  this  is  its  preferred  neighbor  to  which  it  would 
send  any  flow  destined  for  SINK  if  such  flow  comes  in. 

Father:  Node  k is  a father  of  node  i if  node  i is  a son  of  node  k. 

Downstream  node:  Node  t,  is  downstream  from  node  l if  there  is  a set  of 

1 q 

nodes  ,1  . such  that  i.  is  a son  of  l.  for 

. i * 1,2,...,  (q-1) . (see  Fig.  2) 

Upstream  node:  Node  l ^ is  upstream  from  node  if  node  is  downstream 

from  node  J^.  (see  Fig.  2). 

Loop:  A set  of  nodes  * '*'2  * ‘ • • ^orm  a 1°°P  ^ node  is  both 

upstream  and  downstream  from  node  (see  Fig.  2). 

We  are  now  ready  to  describe  the  algorithms.  Each  node  i in  the 
network  has,  for  each  neighbor  k,  memory  locations  called  N^k),  Di(k), 
B^k)  and  R^(k).  N^k)  denotes  a flag  which  can  take  the  value  RCVD  to 
mean  that  a control  message  was  received  at  i from  k.'  during  the  current 
iteration,  or  the  value  NIL  otherwise.  D; (k)  and  B^(k)  are  kept  for 
storing  of  control  messages  received  at  i from  k.  R^(k)  denotes  an 
indicator  which  can  take  the  value  SON  to  mean  that  node  k is  a son 


upstream  direction 
downstream  direction 

d is  a son  of  i . 


Routing,  sons  and  loops. 
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During  the  activity  of  the  protocols  , control  messages  are  sent 

between  neighbors.  These  messages  contain  the  estimated  marginal  delay 

d^  of  the  sender  A, to  the  SINK.  The  control  messages  are  processed  on  a 

FIFO  basis  at  the  receiving  node.  At  first,  the  processor  at  the 

receiving  node,  i say,  identifies  th*  sender, 2 say,  of  the  received 

message,  and  rises  its  N^U)  flag,  i.e.  sets  N^C*)  * RCVD,  then  adds 

to  the  received  d^  the  current  estimated  marginal  delay  D1 on  link 

Ci,JO  and  stores  the  sum  in  D (Z)  • 

1 > 

. 

Suppose  now  that  there  is  a procedure  which  keeps  the  network  loop- 
free  at  all  times.  Each  iteration  of  the  protocols  is  started  when 
the  SINK  enters  state  named  S2,  and  sends  a message  with  dSINK  ■ 0 to 
all  its  neighbors.  Let  us  now  restrict  ourselves  to  an  arbitrary  node 
1 in  the  network  and  describe  its  activities  during  an  iteration  of  the 
protocols.  The  Finite-State-Machine  for  each  node  in  the  network  is 
given  in  Fig.  3.  Generally  speaking,  a node  i enters  the  state  S2  when 
it  has  received  control  messages  from  all  its  sons.  At  this  time  it 
also  updates  its  estimated  marginal  delay  d.,  and  sends  the  updated  d^^ 
to  all  neighbors  except  its  sons.  The  return  to  state  SI  is  performed 
when  the  node  has  received  control  messages  from  all  its  neighbors. 


At  this  time  the  estimated  marginal  delay  dx  is  sent  to  the  sons  and 
routing  changes  are  performed  at  node  i. 
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Before  proceeding  to  explain  the  updating  of  di  and  of  the  routing 
variables  we  describe  the  procedure  of  keeping  the  network  loop-free. 

The  concept  of  blocking  introduced  in  [GALL  77]  is  needed  here.  Briefly, 
if  the  flow  from  node  i over  link  (i,k)  (destined  for  SINK)  is  strictly 
positive  and  d^  i d^,  then  there  is  danger  of  producing  a loop  rn  the 
next  iteration  of  the  algorithms.  To  avoid  this,  if  because  of  the 
constraints  on  the  step-size  involved  in  the  algorithms .node  i is  not 
sure  that  it  can  reroute  all  the  flow  on  (i,k)  in  one  step,  then  it  declares 
itself  blocked,  and  so  do  all  nodes  upstream  from  it.  It  is  shown  in 
[GALL  77]  and  [SEG  77a]  that  loops  are  not  generated  in  the  network  if 
the  following  rule  is  kept:  The  flow  to  a blocked  node  which  is  not  a 
son  is  not  allowed  to  be  increased  from  zero. 

Updating  of  d^  is  done  when  transition  from  state  SI  to  state  S2 
occurs.  We  denote  this  transition  by  T12.  For  the  message  switched  net- 
work d^  is  calculated  using  formula  (2.13).  For  the  line  switched  network, 
d^  is  calculated  as  the  minimum  of  all  D^Ck)  received  by  i up  to  this 
point  from  all  sons  and  other  nonblocked  neighbors.  In  addition,  when 
entering  S2,  node  i updates  its  blocking  status  so  that  any  potential 
loop  will  be  prevented.  Then  it  sends  (the  updated)  d^  and  its  blocking 
status  to  all  neighbors  except  sons. 

While  entering  state  SI,  from  state  S2,  namely  when  transition  T21 
occurs,  node  i reroutes  the  flow  (destined  for  SINK)  in  a particular  way, 
so  that  both  convergence  of  the  protocols  to  the  minimum  delay  routing  and 
the  loop-freedom  property  are  insured.  This  is  done  by  choosing  a pre- 
ferred son,  through  which  the  flow  might  be  increased.  Then  the  routing 
variables  are  changed,  d^  is  sent  only  to  sons  and  then  the  list  of  sons 
is  updated. 
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Notice  that  according  to  the  protocols,  the  updating  of  the  estimated 
marginal  delays  {d^}  propagates  from  the  SINK  upstream  and  the  rerouting 
proper  propagates  downstream  towards  the  SINK.  Clearly,  this  procedure 
is  deadlock  free  if  and  only  if  there  are  no  loops  in  the  network.  We 
see  therefore  that  maintaining  loop-freedom  in  the  network  at  all  times  is 
essential  to  provide  a natural  sequencing  in  the  network,  in  addition  to 
saving  resources. 

Notice  also  that  transition  of  SINK  from  state  S2  to  state  SI 
(remember  the  SINK  enters  S2  when  starting  an  iteration)  means  completion 
of  the  whole  iteration  by  the  entire  network.  The  SINK  is  then  allowed 

to  start  a new  iteration  anytime,  provided  it  is  in  state  SI. 

I 


2.5.3:  Formal  Description  of  the  Protocols 

We  are  now  going  to  display  the  formal  protocols.  First,  we  define 
the  variables  used  by  the  algorithms  at  node  i,  then  the  algorithms 
performed  by  each  node  are  displayed.  At  last,  the  algorithms  performed 
by  SINK  are  presented. 

Since  the  algorithms  for  the  two  introduced  models  are  very  similar, 
we  describe  them  simultanously  and  when  applicable,  indicate  the  differences. 

Definition  of  variables: 

The  values  a variable  can  take  appear  in  parentheses. 

i = node  under  consideration; 

l * the  4-th  neighbor  of  node  i ; Cl  ,2 , . . . ,N) ; 
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n = a parameter:  (see  Theorem  2.2); 

d^  = estimated  marginal  delay  of  node  i from  SINK:  (1,2, . . . ); 

tr  = blocking  status  of  node  i:(0,l);  0 means  not  blocked;  1 means 

blocked; 

MSG  (d,b,Jl)  =■  control  message  received  by  i from  neighbor  Jc : (d=d^ , 

b-6t); 

D'^  = estimated  Cor  calculated)  marginal  delay  on  link  (i,Z)  : (1 ,2, . . .) ; 

d^(i)»  last  number  d received  at  i from  neighbor  Jt:(0,l,...  ); 

N^ (£)  = flag: (NIL,  RCVD);  RCVD  means  a message  has  been  received  at  i 
from  neighbor  l during  the  current  iteration; 

DiU)-  c^CM  * D'iJt  : (1 , 2 , . . . ); 

B^(A)»  blocking  status  of  neighbor  i as  known  at  i : (0 , 1 ) ; 0 means  not 
blocked;  1 means  blocked; 

Ri (A)-  status  of  neighbor  l : (NIL, SON) ; SON  means  node  & is  a son  of  i. 

In  the  formal  description  of  the  actions  done  by  node  i,  we  need 

the  following  sets  of  neighbors: 

= set  of  all  nodes  k such  that  R^(k)  = SON  and  all  other  nodes  k 
with  Fi(k)  = UP,  N.(k)  = RCVD  and  B.(k)  = 0. 

= set  of  all  nodes  k such  that  R^(k)  = SON  and  all  other  nodes 
with  F.OO  » UP  and  B.(k)  ■ 0. 


messages  are  received 
from  all  sons 


messages  are  received 
from  all  neighbors. 


Fig.  3: 


Finite-State-Machine  for  an  arbitrary  node  (Basic  algorithms) 
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The  Algorithms 

(for  each  node  i except  SINK] . 


A. 


88IS&SiSBSaS8BS.&3f.JfcS-Sfe32S8S.252SMSS£.S6ia-S-^2SSi8-ig-BfSSiSSS 


(i.e.  when  the  message  processor  at  node  i takes  MSG(d,b,S.) 
from  the  queue  and  starts  processing  it). 


Execute 

aaasssa 


A.  1 

N. (A)  * RCVD ; 

A. 2 

diCM  - d; 

A. 3 

D.C*)  - d «■  D'u; 

A. 4 

BiCA)  «-  b; 

A. 5 

EXECUTE  FINITE-STATE-MACHINE. 

B. 

Finite-State-Machine 

BSSsasBaBassassaaaaa 

B.l 

STATE  SI 

B.  1. 

1 T12 : Condition 

12:  -V^k  s.t. 

R^Qt]  = SON  , then  N\  (k)  ■ RCVD; 

B.l. 

2 Action  12: 

For  line 

switched  network  set 

d.  min  { D . Qtl > ; (2.15) 

1 k:teCi 

For  message  switched  network  set 

d.  - 7 *ik-D.(k);  (2.16) 

: k:Ri(k)=S0N 
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B.1.3 


B.  1.4 

B.  2 

B.2.1  T21 : 

B.2.2 

B.2.3 

B.2.4 

B.2.4.1 
B.2.4. 2 


Check  of  status:  If  for  any  k s.t. 

R^k)  - SON  then  {BiQc)  ■ 1}  or 
{for  line  switched  network 

d.Ckl  * d.  S n[D.(k)  - dL]  < fik;  (2.17) 

for  message  switched  network 

diOclid.  5 n[Di(Jc)-di]/ti<4iik; } (2.18) 

then  set  b.  *■  1;  otherwise  set  b.  0; 

l 'l 

f k s.t.  F.OO  - UP  and  R^k)  * SON. 
send  (di,bi,i); 

STATE  S2 

Condition  21:  -^-k  s.t.  Fi Ck)  ■ UP,  then  N^k)  ■ RCVD; 

Action  21:  Rerouting: 

Calculate  min  {D.(k)};  (2.19) 

k : keA . 

l 

let  kQ  be  any  neighbor  that  achieves  the 

minimum  in  (2.19); 

For  line  switched  network : 

If  there  is  any  node  q s.t.  F^(ql  with 

f.  > 0,  then  for  all  neighbors  keA. 
lq  i 

do: 

aik  " DiCkl  * a;  C2.20) 

cancel  all  outgoing  links  corresponding 
to  incoming  links,  that  have  been 
cancelled  by  fathers.  Let  f'^k  be  the 
remaining  flows. 
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B.2.4.3 


B.2.4.4 


*ik  " n>in^'ik.n-aik}; 

SET  NEW  FLOW  (*fk  s.t.  F.(k)  ■ UP) 


C2.21) 


keA. ,k^k 
1 o 


if'ik+  l *ik+any  new  flow»k*k0  (2- 22) 

kcA^ 


B.2.4.5 


If  fik  « 0 Vk  s.t.  F.(k)  - UP,  then 
any  new  flow  is  routed  through  k ; 


B.2.5 


For  message  switched  network: 


B. 2.5.1 


B.2.5. 2 


B.2.5. 3 


if  t^  > 0,  then  for  all  neighbors 
keA^  do: 

aik  = DiCk)  ‘ a;  C2>23) 

Aik  • min{<(iik,riaik/ti};  (2.24) 

SET  NEW  FLOW:  (yk  s.t.  F^k)  - UP) 


B.2.5.4 


B.2.6 


B.2.7 


♦r  -i*ik  - 

^ * kL.‘ik 

i 

k^k 

o 

if  t^  = 0,  then  set  4 
^ k i kQ  set  <j>ik  = 0; 


keAi(k?<k0 


k-kQ  C2.25) 


*1  and 


irk  s.t.  Ri Ck)  - SON,  send  (d^b^i); 

Vk  s.t.  FiCk)  - UP,  set  R.(kl  = NIL; 

Set  R.(kQl  * SON;  Set  Ri(k)  - SON 
-k  s.t.  F.  00  - UP,  Mk  and  f"®W>  0 (for 

1 OIK 

ngu 

line  switched  network)  or  4^k  >0  (for 

message  switched  network) ; 
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B.2.8  Vk  s.t.  FiCk)  ■ UP,  set  N.(k)  - NIL; 

This  completes  the  description  of  the  algorithms  for  all  nodes  in 
the  network  except  the  SINK.  The  SINK  performs  the  same  operations  as 
all. other  nodes  and  in  addition,  it  can  start  a new  iteration  at  any 
time,  provided  it  is  in  state  SI,  by  going  into  S2  and  transmitting 
MSG Cd«0,b-0, SINK)  to  all  nodes  k s.t.  FSINKtk)  « UP. 

Finite-S'tate-Machine  for  SINK 

ssi:3S3s3sss3sidtflaianB«isB 

STATE  S2 

T21:  Condition  21:  ^k  s.t.  FSINKCk)  = UP,  then  NSIN!(00-RCVD. 

Action  21:  Tk  s.t.  FSINKCk)  ■ UP,  set  NSJNK(k)  - NIL. 


2.5.4:  Properties  of  the^Protocols 

The  most  important  properties  of  the  two  quasi-static  protocols 
described  before  are: 

Cl)  Distributed  computation  is  used. 

C2)  Loop-freedom  (for  each  destination)  is  maintained  in  the  network 
at  all  times. 

(.3)  Convergence  to  the  minimum  delay  under  certain  conditions. 

The  properties  are  rigorously  stated  in  the  following  theorems, 
whose  proofs  appear  in  [GALL  77],  [SEG  77a], 


Theorem  2. 1 (Loop- Freedom) 

At  all  times,  the  flows  to  each  destination  are  loop-free. 

Theorem  2.2:  (Convergence) 

Let  the  input  traffic  into  the  network  be  stationary,  and  let  the 
topology  of  the  network  be  fixed.  Then  under  assumptions  (2. a),  there 
is  a sufficiently  small  value  of  the  parameter  n such  that  converges 
to  the  value  of  the  minimum  average  delay  over  all  routing  assignments, 
for  any  initial  flow. 

2.5.5:  Initialization_of_the  Protocols 

Obviously,  the  protocols  must  be  started  with  some  loop-free  flow. 
The  following  starting  rule  is  suggested  in  [SEG  77a] : When  the  network 
starts  operating,  each  node  chooses  its  son  to  be  the  first  node  from 
which  it  receives  a control  message.  Thus  an  upstream  relation  is 
established  and  the  algorithms  can  continue  as  usual.  This  topic 
is  further  discussed  in  the  next  chapter  when  addition  of  links  to  the 
network  is  taken  into  account. 
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CHAPTER  3 


ROUTING  PROTOCOLS  FOR  NETWORKS  WITH  CHANGING  TOPOLOGY 

— 

. 

3.1:  Introduction 

The  protocols  described  in  Chapter  2, ref erred  to  from  now  on  as 
the  basic  protocols,  can  operate  smoothly  only  when  no  topological 
changes  occur  in  the  network  and  in  that  case,  they  gradually  adapt  to 
changes  in  the  traffic  requirements.  However,  since  nodes  and 
communication  links  occasionally  fail  and  recover  in  any  practical 
network,  the  basic  protocols  should  be  expanded  to  handle  arbitrary 
topological  changes,  while  preserving  the  main  properties  of  the  basic 
protocols.  The  protocols  presented  in  this  chapter  are  designed  to 
do  so,  independently  of  the  number,  timing  and  location  of  those 
topological  changes.  These  protocols  are  a natural  extension  of  the 
protocol  of  [SEG  77c]  where  a single  optimal  route  was  maintained  from 
each  node  to  the  destination.  Essentially,  the  extra  feature  provided 
by  the  pfesent  protocols  compared  to  the  protocol  of  [SEG  77c]  is  to 
indicate  the  exact  amount  of  flow  splitting  so  that  optimal  average 
delay  is  obtained  in  the  network  in  steady- state.  As  such,  our 
resulting  protocols,  to  be  presented  in  the  subsequent  sections,  have 
all  of  the  following  properties: 

(1]  Distributed  computation  is  used. 

(.2)  Loop-freedom  for  routes  for  each  destination  is  maintained  at  all 
times. 

(3)  Recovery  of  the  network  in  finite  time  from  arbitrary  number,  timing 
and  location  of  topological  changes. 


(4)  If  the  traffic  is  stationary  and  the  topology  fixed  for  long  enough 

time,  the  network  is  brought  in  steady-state  to  the  minimum  delay  routing. 


40 


In  general,  the  description  of  these  protocols  follows  the  same 
pattern  as  for  the  basic  protocols.  The  main  basic  changes  are  that  the 
finite-state-machine  contains  more  states  and  the  control  messages 
contain  more  information.  The  entire  extension  is  given  in  the 
subsequent  sections. 

To  analyse  the  protocols  and  validate  their  properties  and  correctness, 
a technique  introduced  in  [SEG  77c]  is  used.  According  to  this  technique, 
a special  type  of  induction  is  used,  that  allows  to  prove  global 
properties  while  essentially  looking  at  local  events.  The  main  proofs 
are  given  in  the  appendices. 

The  extension  of  the  basic  protocols  is  exactly  the  same  for  both 
message  and  line  switching.  Consequently,  in  the  informal . description 
we  do  not  distinguish  between  the  two  models,  and  return  to  do  so  only 
in  the  formal  description.  In  the  last  section  of  this  chapter  all 
properties  of  the  resulting  protocols  are  formally  stated  in  a series 
of  theorems,  whose  proofs  appear  in  the  appendices. 


3.2:  Informal  Description  of  the  Protocol 

3.2,1:  Introduction 

The  operations  to  be  performed  by  the  algorithm  at  each  node  in 
"normal"  conditions,  namely  when  no  topological  changes  occur  m the 
network. have  been  described  in  the  basic  protocols.  Here  we  first  give 
the  general  additions  to  the  protocol  and  then  the  details  are  provided, 
first  for  link  failures  and  then  for  links  becoming  operational.  We  do 
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not  pay  special  attention  to  topological  changes  caused  by  nodes,  since 
such  changes  might  be  perceived  as  the  change  of  status  of  all  links 
connected  to  those  nodes. 

This  protocol  is  still  operating  independently  for  each  destination 
and  as  before,  we  present  it  for  a given  fixed  destination  called  SINK. 

3.2.2:  General_Additions 

For  later  purposes,  there  is  need  to  number  the  interations  of  the 
protocol  with  nondecreasing  numbers  as  described  below.  Each  node  i will 
have  a node  counter  number  n^  which  denotes  the  iteration  number  currently 
handled  by  this  node.  All  control  messages  transmitted  by  i will  carry  n^ 
in  addition  to  d.  and  b.,  namely  they  will  be  of  the  form  MSGCm,d,b,i) 
with  m»n^,  d^d^  and  b^b^  When  a MSGQn,d,b,£)  is  received  by  node  i on 
link  then  d and  b are  stored  in  D^C£)  and  B^(£)  respectively,  as 

dictated  by  the  basic  algorithms,  and  in  addition  there  is  also  need  to 
remember  the  value  of  m.  This  value  can  be  saved  in  N.(£),  which  can  now 
take  the  values  NIL,  0,1,2,...;  instead  of  NIL  and  RCVD  as  for  the  basic 
algorithms.  For  simplicity,  the  parameter  l in  MSG(m,d,b,a)  is  suppressed 
from  now  on. 

Generally  speaking,  after  having  received  MSG(m,d,b)  with  a given 
counter  number  m from  all  its  current  sons,  node  i updates  its  counter 
number  n^  to  m and  effects  transition  T12  in  the  same  way  as  for  the 
basic  algorithms.  Transition  T21  is  performed  when  the  node  receives 
MSG(m,d,b}  with  counter  number  m from  all  its  current  neighbors. 
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In  our  extended  protocols  the  SINK  starts  consecutive  iterations 
with  nondecreasing  counter  number.  If  the  SINK  starts  an  iteration  with 
counter  number  m = n^^  and  completes  it  before  starting  a new  iteration, 
we  say  that  there  has  been  a proper  completion.  IVe  denote  the  time  of 
proper  completion  of  an  iteration  with  number  m by  PC(m).  In  this 
case,  the  SINK  is  allowed  to  start  a new  iteration  with  the  same  counter 
number. 


To  handle  topological  changes,  there  are  situations  that  the  SINK 
must  increase  the  iteration  counter  number.  The  protocol  allows  it  to 
do  so  at  any  time,  whether  the  previous  iteration  was  completed  or  not. 
(Notice  that  in  any  case  the  values  of  n^j^  are  nondecreasing  with  time). 
As  proved  later,  if  a new  iteration  is  started  while  increasing  ngINK>  it 
will  eventually  cover  all  previous  iterations. 


There  are  several  possible  ways  to 
its  (and  starts-  an  iteration  with 
after  need  arises.  These  possibilities 
4,  but  for  the  purposes  of  this  chapter 
an  algorithm  indeed  exists.  The  formal 
A in  Section  3.4.3. 


insure  that  the  SINK  increases 
this  number)  finite  time 
are  described  in  detail  in  Chapter 
it  is  enough  to  assume  that  such 
assumption  is  given  in  Assumption 


3.2.3:  Handling  Failures  of_Links 

We  now  turn  to  describe  the  algorithm  for  a node  i that  discovers  a 
failure  on  one  of  its  incident  links.  We  assume  here  (see  formal  assumptions 
7,9,  in  Section  3.3.2)  that  whenever  link  (i,H)  fails  then  link  (J.,i) 
fails  at  the  same  time,  but  the  nodes  i and  l may  discover  the  failure 
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at  different  instants.  However,  if  i discovers  a failure  onCi,*),  it 
cannot  bring  the  link  up  before  l discovers  the  failure  too. 

There  are  three  typical  situations  to  he  distinguished.  First, 
the  case  when  the  node  has  only  one  son  and  discovers  a failure  on  the 
link  to  this  son.  Second,  the  situation  when  the  node  has  more  than 
one  son,  and  the  failure  is  discovered  on  the  link  to  one  of  its  sons. 
Third,  the  case  when  the  failure  occurs  on  a link  to  a neighbor  that  is 
not  a son.  In  all  these  cases,  the  first  action  node  i takes  when 
discovering  a failure  on  link  (i.^)  is  to  set  Fi(Z)  * FAIL,  thus 
indicating  that  node  i is  not  a neighbor  any  longer.  Now,  the  role 
of  F^(£)  becomes  apparent.  F^C*0  indicates  the  status  of  link  (i,2) 
as  seen  from  node  i.  F^*)  = UP  means  that  the  link  Ci.^J  is  under 
normal  operation,  namely  that  l is  a neighbor  of  i.  Fi(Z)  = DOWN 
means  that  the  link  (i,A)  is  unoperational . F^Jl)  can  also  take  the 
value  READY  whose  use  will  become  apparent  when  dealing  with  links 
becoming  operational. 

Single  Son 

If  node  i has  only  one  son,£  say,  and  link  Ci.£)  fails,  then 
node  i loses  its  only  route  to  the  SINK.  In  addition,  some  nodes 
upstream  from  node  i lose  one  or  more  of  their  routes  to  the  SINK. 
However,  all  those  upstream  nodes  are  unaware  of  this  fact  at  the 
time  the  failure  occurs.  For  instance  (.see  Fig.  4),  if  link  C6,l) 


fails  then  nodes  6, 8, 5,9  lose  all  their  routes  to  the  SINK  and  nodes 
4,7  and  10  lose  one  of  their  routes  to  the  SINK.  Furthermore,  if  an 


44 


iteration  is  started  by  the  SINK,  node  6 will  never  be  able  to  receive 
a control  message  from  node  1,  and  therefore,  node  6,  as  well  as  nodes 
4, 5,7, 8, 9 and  10,  will  never  be  able  to  perform  T12.  The  extentions  to 
the  basic  algorithms  provided  here  are  designed  to  allow  the  network  to 
recover  from  this  situation,  namely  to  provide  alternate  routes  to  nodes 
that  have  lost  their  only  route  to  the  destination  and  to  allow  the 
other  affected  nodes  to  continue  their  normal  operation  This  and  the 
next  subsections  indicate  these  actions. 

Two  actions  must  be  taken  by  the  extended  protocol.  First,  to  inform 
the  nodes  upstream  from  node  i not  to  wait  for  control  messages  from 
their  sons  that  are  on  the  failed  routes,  and  also  to  notify  them  that 
the  routes  do. not  exist  any  longer  (e.g.  in  Fig.  4 node  8 should  be 
informed  that  the  path  8,6,1,  SINK  does  not  exist  any  longer).  Second, 
to  allow  node  l (and  possibly  its  upstream  nodes  that  lost  all  their 
routes)  to  choose  a new  son  whenever  control  messages  of  new  iterations 
will  be  received  This  features  are  described  presently. 

Whenever  a node  i discovers  a failure  on  link  (i,£),  where  i is  its 
only  son,  it  sets  R^Ci.)  = NIL  and  d^  = » to  mean  that  j,  is  no  longer  its 
son  and  that  its  marginal  delay  to  the  SINK  has  become  infinite.  Then 
node  l generates  a special  control  message,  MSG(n^,d=«,b^j  which  is 
sent  to  all  neighbors  of  i except  1;  if  a node  k receives  such  a message 
from  its  only  son,  q say,  then  it  performs  similar  operations,  namely  it 
sets  R^Cq)  * NIL,  d^  = 00  and  sends  MSG(n^,d=“,bj<)  to  all  its  neighbors 
except  node  q;  if  a node  receives  such  a message  from  a neighbor  that  is 
not  a son,  it  stores  it  but  no  other  action  is  taken;  the  case  when  the 
node  has  more  than  one  son  and  such  a message  is  received  from  some  son 
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is  discussed  in  the  next  subsection.  When  a node  i establishes 
Ri  (i)  = NIL  for  its  single  son  and  = °°,  it  also  enters  state  S3. 

A node  that  enters  state  S3  must  select  a new  son,  thus  establishing 
a new  route  to  the  SINK.  This  procedure  is  the  second  part  of  the  recovery 
and  is  called  reattachment . The  reattachment  takes  place  if  one  of  the 
following  two  situation  occurs . One  possibility  is  when  a node  i in  state 
S3,  (and  hence  with  no  sons),  receives  a control  message  MSG(m,d*°°,  b) 
from  l say,  with  m > n^.  Then  node  i knows  that  this  message  was  generated 
by  an  iteration  started  after  the  failure  that  caused  its  entrance  to 
state  S3.  A second  possibility  is  that  such  a message  has  already  been 
received  at  i from  l at  the  time  i enters  state  S3.  The  reattachment 
consists  of  setting  = SON,  going  to  state  S2  and  effecting  the  same 

operations  as  in  T12.  This,  together  with  other  procedures  to  be  presented, 
guarantees  that  if  any  number  of  failures  occur,  and  if  the  SINK 
realV  starts  an  iteration  that  cover  all  these  failures  (as  we  assumed) 
i.e.  an  iteration  with  counter  number  that  was  not  the  node  number  of  any 
of  the  nodes  detected  the  failures  while  detection,  then  each  node 
physically  connected  to  the  SINK  will  eventually  have  at  least  one  route 
to  the  SINK.  Furthermore,  no  loops  are  generated  by  the  reattachment 
procedure,  a property  that  is  stated  in  Theorem  3.1  and  proved  in 
the  appendices.. 

More  than  one  Son 

If  node  i has  more  than  one  son  and  a failure  is  discovered  by  node 


i on  link.  (i,*0  connecting  it  to  one  of  its  sons,  l say,  or  if  node 
i receives  MSG(m,d»®,b)  from  l,  then  node  i knows  that  its  route  to 
the  SINK  passing  thourgh  1 has  been  destroyed  (e.g.  failure  on  link  (4,5) 
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in  Fig.  4 detected  by  node  4],  Naturally,  must  be  set  to  NIL 

to  indicate  that  l is  not  a son  any  longer,  but  no  transition  should 
be  performed  and  no  special  action  is  to  be  taken,  since  node  i still 
has  other  sons.  .This  is  accomplished  by  introducing  Cl  (and  later  also 
C2)  into  the  Finite-State-Machine,  in  which  the  only  action  taken  is  to 
set  R^(L)  a NIL,  while  staying  in  the  same  state.  However,  if  node  i 
is  in  state  S2,  it  is  necessary  to  prevent  T21  from  happening  at  this 
node  for  the  current  iteration,  the  reason  being  that  this  will  prevent 
nodes  from  updating  their  routes  based  upon  information  which  is 
invalidated  by  the  failure.  We  rather  explain  the  last  sentence.  While 
entering  state  S2  node  i calculates  its  marginal  delay  d^  to  the  SINK 
and  determines  its  blocking  status  b^ . Suppose  that  while  being  in  S2, 
the  link  to  the  node  that  the  calculation  of  d^  was  based  upon,  fails. 
Then  there  is  a danger  that  its  blocking  status  b^  is  incorrect.  There- 
fore, if  T21  will  be  performed,  a loop  might  be  generated  - a situation 
we  must  avoid.  Another  important  reason  is  that  prevention  of  T21  from 
happening  will  also  preclude  proper  completion  from  happening.  Thus, 
proper  completion  will  now  indicate  to  the  SINK  that  the  iteration  was 
completed  without  failures  interfering  with  the  process.  Prevention  cf 
T21  is  accomplished  by  introducing  an  additional  state,  S2,  into  which 
a node  enters  if  it  has  more  than  one  son  and  either  detects  a failure 
on  a link  connecting  it  to  one  of  its  sons,  or  receives  MSG (m,d*“,b) 
from  it,  while  being  in  state  S2.  A node  i will  leave  S2  whenever  it 
receives  new  control  messages  from  all  its  sons.  To  be  more  specific, 
node  i leaves  S2  by  either  going  to  state  S3  in  case  a failure  is  sensed 
its  single  route  to  the  SINK  (as  we  have  already  described) , or  going 
to  state  S2  when  receiving  MSG(m,dj<®,b)  of  a new  iteration,  i.e.  m > n^, 
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from  all  its  sons,  and  in  this  case  it  effects  the  same  operations  as 
in  T12 . 

The  intention  of  the  part  of  the  algorithm  described  in  the  last 
two  subsections  is,  to  enable  upstream  propagation  of  the  knowledge  of  a 
failure  qccurrence .All  nodes  that  are  upstream  from  the  failure  are 
informed  that  they  cannot  send  any  flow  through  the  failed  routes.  In 
addition,  neighbors  of  the  nodes  that  have  lost  all  their  routes  to  the 
SINK,  are  informed  not  to  choose  these  nodes  as  their  sons. 

Clearly,  in  addition  to  all  the  above  operations,  each  node  that 
has  lost  one  of  its  sons  should  stop  the  flow  to  that  son,  transmit  it 
through  its  remaining  sons,  if  it  still  has  any,  and  modify  its  routing 
variables  correspondingly.  The  question  of  how  should  the  node  distribute 
the  flow  between  its  remaining  sons  is  immaterial  for  the  purposes  of  our 
work:  This  is  because  a failure  usually  causes  dramatic  changes  in  the 
routing  variables,  thus  in  the  total  delay,  so  the  exact  distribution 
is  unimportant  in  our  quasi-static  algorithm  that  allows  only  fine 
changes.  We  may  assume  that  it  is  done  in  some  way  that  insures  that 
the  capacity  constraints  are  not  violated.  Another  open  question  is 
what  should  the  node  do  with  the  flow  if  it  has  no  route  to  the 
destination.  In  this  case  we  may  assume  that  it  stores  the  flow  until 
it  establishes  a new  route,  if  it  can,  otherwise,  it  rejects  it. 

Failure  of  a neighbor  that  is  not  a son 


Up  to  now  we  have  described  the  algorithm  for  a node  i discovering 
a failure  on  a link  connecting  it  to  one  of  its  sons.  If  failure  occurs 
on  a link  to  a neighbor  which  is  not  a son,  fe.g.  link  (.6,3)  in  Fig.  4), 


then  no  route  is  disrupted,  so  no  special  action  is  needed.  However, 
for  reasons  explained  in  the  previous  subsection,  if  the  failed  link  is 
connected  to  a node  in  state  S2,  it  is  convenient  here  also  to  prevent 
T21  from  happening  at  this  node  for  this  iteration.  Consequently,  a 
node  enters  S2  whenever  a link  to  nonson  fails  while  the  node  is  in 
state  S2.  The  procedure  of  leaving  S2  has  already  been  described. 

The  protocol  as  described  up  to  now  is  implemented  by  the  algorithm 
in  the  formal  description  given  in  Section  3.5.3  if  ignoring  steps  A. 2, 

A. 2.1  - A. 2. 4,  A. 3.1,  B.1.7,  B.2,8,  B.8.8.  These  steps  relate 

mainly  to  links  becoming  operational  and  will  be  discussed  in  the 
subsequent  section.  The  notations  used  here  are  similar  to  those  in 
chapter  2.  A summary  of  these  notations  is  given  in  Subsection  3.3.1. 
There,  the  variables  used  by  the  algorithm  performed  by  an  arbitrary  node 
i as  its  part  of  the  protocol,  are  given.  F^(£)  denotes  the  status  of 
link  (.i.Ji.)  as  considered  by  node  i,  namely  F^OO  = UP  if  i is  considered 
operational  and  F^i)  = DOWN  if  l is  considered  unoperational . F^(£) 

can  also  take  the  value  READY,  whose  use  will  become  clear  when  dealing 
with  the  problem  of  links  becoming  operational.  At  that  time  the  role 
of  Z^(A)  will  also  become  apparent.  The  variable  mx.  stores  the 
value  of  the  largest  counter  number  m of  all  messages  MSG(m,d,b)  received 
by  node  i from  all  its  neighbors. CT  plays  a role  of  a flag  indicating 
the  number  of  transitions  that  have  already  been  performed  by  the 
Finite-State-Machine,  triggered  by  the  current  message,  0 will  mean  zero 
transitions,  1 will  mean  one  or  more  transitions.  The  rest  of  the 
variables  and  their  use  were  already  described.  The  link  (local)  protocols 
controlling  the  operations  of  the  links  connected  to  node  i may  relay  to 
the  algorithm  performed  by  node  i three  types  of  messages.  MSG  denotes  an 
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updating  message,  FAIL(Z)  denotes  the  detection  of  the  failure  of  link 
(i,£).  The  remaining  message  WAKEOl  is  described  later. 

To  give  here  a short  summary,  remember  that  states  SI  and  S2  and 
transitions  T12  and  T21  are  similar  to  those  described  in  the  basic 
algorithms.  State  S3  denotes  the  situation  where  a node  i has  R^OOaNIL 
V k s.t.  F^(k]  ■ UP  which  results  from  receiving  FAIL  or  MSG  with  d ■ =° 
from  a single  son.  State  S2  denotes  a state  similar  to  S2,  but  from 
which  a transition  T21  is  not  allowed.  As  previously  described,  a node 
goes  to  such  a state  S2  if  while  at  S2,  either  a FAIL  or  a MSG  with  d = » 
is  received  from  a nonsingle  son,  or  if  a FAIL  is  received  from  a nonson 
neighbor.  Transition  T22  is  performed  by  node  i when  control  messages 
MSG(m,d,b)  with  m > n^  are  received  from  all  its  sons.  The  operations 
effected  in  T22  are  the  same  as  in  T12.  Cl  and  C2  are  not  transitions.  In 
Cl  and  C2,  the  action  that  is  taken  is  to  cancel  one  route  to  the  SINK 
while  being  in  state  SI  and  state  S2,  respectively  (see  Fig.  5). 

3.2.4:  Handling  Links  Becoming  Operational 

If  link  Ci,£)  is  down,  namely  F.(£)  ■ F (i)  = DOWN,  and  it  becomes 

2.  A* 

operational,  nodes  i and  i should  coordinate  the  necessary  operations 
to  bring  the  link  up.  Otherwise,  a deadlock  can  occur.  For  instance, 
suppose  node  i sets  F. (2.)  = UP  while  at  state  S2  and  node  l sets  F0(i)  =»  UP 
after  performing  T21  of  the  same  iteration.  In  this  case,  node  i will 
not  perform  T21  until  receiving  a control  message  from  node  i , and  such 
a message  will  not  be  sent  because  l has  already  completed  this  iteration, 


i.e.  deadlock. 
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The  coordination  is  achieved  by  having  botn  nodes  bring  the  link  up 
just  before  starting  to  perform  their  part  of  the  same  new  iteration.  This 
is  done  as  follows:  When  nodes  i and  l sense  that  link  (i,A)  becomes 
operational,  they  compare  their  node  counter  numbers,  n^  and  n^,  via 
their  link  (local)  protocol,  and  decide  to  bring  up  the  link  when  starting 
to  process  the  first  iteration  with  a number  strictly  higher  than  max(n, ,n  j. 
This  fact  must  be  remembered  at  the  nodes  and  it  is  done  be  setting  the 
memory  locations,  Z.(£)  at  i and  Z (i)  at  l,  to  max(n . ,n0 } . Clearly, 

{Z^Ck))  are  memory  locations  kept  at  i for  each  possible  neighbor  k of  i. 

In  addition  to  the  above  operations,  nodes  i and  k also  set  F^fA)  and 
F;(i)  to  READY,  and  N^(i)  and  N^i)  to  NIL.  In  order  to  bring  the  link  up, 
there  is  need  that  the  SINK  will  start  an  iteration  with  nSI^K  larger  than 
Z^OO  (and  Z^Ci)).  By  the  assumption  we  made  in  Section  5.2.2,  such  an 
iteration  is  indeed  started  in  finite  time. (See  also  Assumption  A in  Sec.  3.4. 3) 


The  execution  of  the  first  step  of  the  coordination,  at  node  i is 
triggered  by  a special  control  message  - WAKE (A ) given  by  the  link  (local) 
protocol  to  the  algorithm  at  noe  i (and  similarly  WAKE(Jl)  is  delivered  to 
the  algorithm  at  node  A).  The  actions  performed  by  the  algorithm  when 
receiving  such  a message  are  described  in  A. 2,  A. 2.1  - A. 2. 4 in  the 
formal  description  in  Section  3.3.3.  This  synchronization  assumes  that 
the  execution  of  WAKE(A)  and  WAKE(i)  are  simultaneously  started  at  nodes 

l 

i and  l respectively,  in  order  to  guarantee  that  Z . (Jl)  * Z,(i).  However, 
it  may  happen  that  a failure  occurs  again  on  the  link  and  one  of  the  nodes 
succeeds  to  complete  the  synchronization  while  the  other  node  does  not. 

The  protocol  allows  for  such  a situation  and  only  requires  that  the  link 
protocol  ends  the  synchroniation  (successfully  or  unsuccessfully)  within 
finite  time.  If  the  synchronization  is  unsuccessful,  no  action  is  taker,  sy 
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the  node,  and  the  link  will  remain  DOWN  from  this  node's  point  of  view 
Section  3.3,2  gives  a more  formal  and  complete  list  of  the  requirements 
that  the  link  (.local)  protocol  should  satisfy. 

The  link  (i,£)  is  finally  brought  up  by  node  i,  namely,  F^(f)  is 
set  from  READY  to  UP,  when  node  i receives  MSG  from  link  (i,>.),  or 
when  the  node  counter  number  n becomes  larger  than  Z^Ci). 


3.2.5:  The_Algorithm  for  the  SINK 


The  algorithm  for  the  SINK  is  similar  to  that  for  an  arbitrary  node 
i,  except  that  the  SINK  does  not  need  to  keep  the  following  variables: 


rsinkCJI] 


dsink^j 

mXSINK 

ZSINKW 


(which  is  not  defined  for  the  SINK,  since  it  has  no 
sons) . 

(which,  is  0 by  definition  for  the  SINK). 

(which  is  0 by  definition  for  the  SINK,  since  it 
has  no  sons) . 

(which  is  only  needed  to  update  and  ^INK^^ 

(nSINK  1S  always  t*ie  largest  update  counter  number} 
(during  WAKE  synchronization  iS  a^wa/s  Set 

t0  nSINK  “ maxtnSINK’ni:} ' ^ 


In  addition,  the  SINK  can  start  a new  iteration  at  any  time,  by  gcin,; 
to  state  S2  and  sending  MSG^g^^.dsO ,b=Oj  to  all  its  current  neighbors, 
provided  that  the  last  iteration  was  propertly  completed.  Moreover,  when 
necessary,  the  SINK  increments  its  ngINK  and  starts  a new  iteration  a^ 
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described  above,  even  if  the  last  iteration  was  not  properly  completed. 
We  have  seen  that  there  is  need  to  increment  and  start  a new 

iteration  whenever  a topological  change  occurs  in  the  network.  The 
exact  details  of  how  this  can  be  actually  done  in  a distributed  way  are 
provided  in  the  next  chapter. 

In  the  algorithm  for  the  SINK,  states  SI  and  S2  are  similar  to  the 
corresponding  states  of  the  algorithm  for  an  arbitrary  node  i.  However, 
for  the  SINK,  SI  means  that  the  last  iteration  was  properly  completed, 
and  S2  means  that  the  last  iteration  is  not  yet  completed.  T12  and  T22 
represent  the  starting  of  a new  iteration  and  T21  represents  proper 
completion  (see  Fig.  61.  For  the  SINK  there  is  no  need  for  states 
equivalent  to  S3  and  S2  of  the  algorithm  for  an  arbitrary  node  because 
whenever  the  SINK  detects  a topological  change  it  starts  immediately  a 
new  iteration  while  incrementing  n^^. 

3.2.6:  Initialization  of  the  Protocol 

A node  i comes  into  operation  in  state  S3,  with  node  counter  number 
n^  ■ 0,  and  R^(k)  * NIL,  F^(k)  ■ DOWN  for  all  k.  The  value  of  the 
remaining  variables  is  immaterial.  From  this  initial  conditions,  the 
link  (local)  protocol  may  try  to  wake  the  links  and  it  proceeds 
operating  as  defined  by  the  algorithm.  The  SINK  comes  into  operation 
in  state  SI,  with  n2j^K  = ® ans*  ^SINK^  “ DOWN  f°r  all  k,  and  proceeds 
according  to  the  algorithm  for  the  SINK. 
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3.3:  Formal  Description  of  the  Algorithms 

5.3.1:  Introduction 

We  are  now  ready  to  display  the  formal  algorithms  performed  by  each 
node  i in  the  network.  As  for  the  basic  algorithms , we  present  here  the 
algorithms  for  the  two  models  (message  switching  and  line  switching) 
simultanously  and  indicate  the  differences  when  applicable.  The  presentation 
here  follows  the  same  lines  as  the  basic  algorithms.  In  addition,  in 
Section  3.3.2  we  provide  the  exact  requirements  from  the  local  (link) 
protocol.  The  "Facts"  given  in  the  algorithms  are  displayed  for  helping 
in  their  understanding  and  are  proven  in  Theorem  3.2  of  Section  3.4.3. 

A Fact  holds  if  the  transition  under  which  it  appears  is  performed. 

Definitions  of  variables 

The  values  a variable  can  take  appear  in  parentheses. 

i * node  under  consideration. 

I = the  i-th  neighbor  of  node  i:  (1,2,... ,N); 
n ■ a parameter:  (see  Theorem  2.21; 

n^  ■ current  counter  number  of  node  i : (0, 1 ,2 , . . . ) ; 
di  = estimated  marginal  delay  of  node  i from  SINK:  (1,2,...,®); 

bi  ■ blocking  status  of  node  i : (0, 1) ; 0 means  not  blocked;  1 means 

blocked; 

The  processor  at  node  i may  receive  the  following  types  of  messages  related 
to  each  link  (i,£): 

MSG(m,d,b,A)  ■ updating  message  received  by  i from  f:  (m^n^  ,d=di- ,b=b^) ; 

FAIL(A)  =>  failure  detected  on  link  (i,£); 
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WAKEiA)  = link  (i,A)  becomes  operational,  i.e.  messages  can  be  sent 
through  it; 

We  now  continue  the  list  of  variables: 

F^(A)  3 status  of  link  (i,A)  as  seen  from  node  i : QJP , DOWN,  READY);  UP 

means  the  link  is  operational;  DOWN  means  the  link  is  unoperational ; 
READY  means  the  link  is  ready  to  be  brought  up; 

N\(A)  = the  number  m received  from  neighbor  A during  the  current 
iteration : (NIL,  0,1,...); 

D’.  * estimated(or  calculated)  marginal  delay  on  link  (i,A):  (1,2,...); 

di  (A)  = last  number  d received  at  i from  neighbor  A : (0, 1 ,2 ,...,») ; 

D.  (A)  = di(i)  ♦ D'u:  (1,2,.  . . ,»); 

3.  (A)  » blocking  status  of  neighbor  A as  known  at  i : (0, 1) ; 0 means  not 
blocked;  1 means  blocked; 

R.fA)  =*  status  of  neighbor  A:  (NIL,  SON);  SON  means  node  A is  a son  of  i; 

i 

Z . (A)  = a synchronization  number  indicating  the  iteration  number  upon  which 
the  link  (i,A)  can  be  brought  up,  i.e.  changed  from  READY  status 
to  UP  status : (0, 1 , 2, ...) ; 

mx^  ■»  the  largest  number  m received  by  node  i up  to  the  current  time 
from  all  neighbors : (0, 1 ,2 ,...) ; 

CT  = a flag  indicating  the  number  of  transitions  the  Finite-State- 

* 

Machine  has  already  performed  triggered  by  the  current  -message: 
(0,1);  0 means  zero  transitions;  1 means  one  or  more 
transitions . 

In  the  formal  description  that  follows,  we  will  need  to  refer  from 
time  to  time  to  certain  sets  of  neighbors.  To  save  space,  we  define  those 


sets  here: 


C = set  of  all  nodes  k s.t  R.  (k)  = SON  and  all  other  nodes  k with 

1 i 


F.  (_k)  = UP,  N.(k)  = mx^  and  B± CkJ  = 0. 

A^  * set  of  all  nodes  k s.t.  (k } = SON  and  all  other  nodes  with 
FiCk}  « UP  and  B (k)  = 0. 


3.3.2:  Properties  Required  from  the  Local  Protocol 

On  each  link  of  the  network  there  is  a link  (.local)  protocol  that 

is  in  charge  of  exchanging  messages  between  neighbors.  Our  main  algorithm 

assumes  that  the  following  properties  hold  for  the  local  protocol: 

1.  All  links  are  bidirectional  (duplex). 

2.  D'.,  > 0 for  all  links  (i,k)  at  all  times.  (See  (2  9d)j 

lk 

3.  If  a message  is  sent  by  node  i to  a neighbor  4. , then  in  finite 
time,  either  the  message  will  be  received  correctly  at  l or 
F^Ul  = F ^ Ci  1 = DOWN.  Observe  that  this  assumption  does  not 
preclude  transmission  errors  that  are  recovered  by  the  local  protocol 
(e.g.  "resend  and  acknowledgement"). 

4.  Failure  of  a node  is  considered  as  failure  of  ail  links  connected 
to  it. 

5.  A node  i comes  up  in  state  S3,  with  nx  = 0,  Ri(k)  = NIL  and 
Fi(k)  = DOWN  for  all  links (i,k). 

6.  The  processor  at  node  i receives  messages  from  link  ( i , A ) on  a first- 
in-first-served  (FIFO)  basis. 

7.  A link  (i,Jl)  is  said  to  have  become  operational  as  soon  as  the  local 
protocol  discovers  that  the  link  can  be  used.  Links  (i,£)  and  (L,i) 
become  operational  at  the  same  time  and  subject  to  the  following 

L; - • ---  ‘ : J 
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restrictions , a WAKE  "message"  is  delivered  in  this  case  to  each 
of  the  processors  i and  i. 

WAKE (2 ) can  be  received  at  node  i only  if 

Ca)  node  l receives  WAKE(i]  at  the  same (virtual)  time; 

(b)  there  are  no  other  outstanding  messages  on  link  (i,Jl)  and 
on  C£,i); 

(c)  F .(*)  = F4(i)  = DOWN. 

8.  If  F^OO  = DOWN,  the  only  message  that  the  processor  at  i can 
receive  from  i is  WAKE(i). 

9.  Ca)  If  F.  00  * DOWN  and  F0Ci)  / DOWN  and  F.  00  goes  to  DOWN,  then 

F^Ci)  goes  to  DOWN  in  finite  time. 

(b)  If  F^i)  = Fj^Ci)  ■ DOWN  and  F^i.)  goes  to  READY,  then  in  finite 
time,  either  F^Ci)  goes  to  READY  or  F^OO  =*  F^(i)  = DOWN. 

10.  When  two  nodes  i and  l receive  WAKE  as  described  in  7,  a 

"synchronization"  between  i and  l is  attempted.  At  either  end  the 
synchronization  may  or  may  not  be  successful  Cthe  latter  because 
of  a new  failure).  If  it  is  successful,  the  node  proceeds  as  in 
Step  A. 2 of  the  formal  description.  If  not,  then  no  action  is 
taken. 
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3.3.3.:  Formal_Algorithms 


(For  each  node  i except  the  SINK) . 


A.  Operations  Done  by  the  Message  Processor  when  a Message  is  Received 

(i.e.,  when  the  message  processor  at  node  i takes  the  message  from 
the  queue  and  starts  processing  it). 


A.  1 For  FAILU) 


A.  1.1  F.  (A)  - DOWN; 

A. 1.2  CT  - 0; 


A.  1.3  EXECUTE  FINITE-STATE -MACHINE . 


A. 2 For  WAKE (A) 


(Fact:  F^(£)  = DOWN,  see  7 in  Section  3.3.2) 

A. 2.1  wait  for  end  of  WAKE  synchronization, (see  10  in  Section  3.3.2)  if 
WAKE  synchronization  is  successful,  then 
A. 2. 2 Z^)  +-  maxfn^n  }; 

A.  2. 3 F . (A)  <-  READY; 

A.  2. 4 Ni(A)  NIL. 


A. 3 For  MSG  (jn,d,b,A) 


.3.1  if  F.  U)  * READY,  then  F.Qll 


(Fact:  m > Z^A)]; 


A. 3. 2 Ni (4)  «-  m; 

A.  3. 3 di(i)^d; 

A.3.4  D.(A)  - d ♦ D'u; 

A.3.S  BiU)  «■  b; 

nu^  *■  maxOn.mx^ ; 


A. 3. 6 
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A. 3. 7 CT  - 0; 


A. 3.8  EXECUTE  FINITE -STATE -MACHINE 


B.  FINITE-STATE-MACHINE 


STATE  SI 


B.1.1  T12  Condition  12:  yk  s.t.  R.Qt}  = SON,  then 


B.1.2 


B.  1.3 


B.  1.4 


Fact  12 


Action  12 


Di(Jc)  / - and  F.(k)  = UP; 
CT  * 0 


If  MSG,  then  m i n^ . 

For  line  switched  network  set 


d • min  {D . (k) } : 
k:keC.  1 

l 

For  message  switched  network  set 


B.  1.5 


di  l $ .D  Ck); 

k:R.(k)=SCN 

l 

Check  of  status:  If  for  any  node  k 


s.t.  R^(k)=S0N  then  {B^(k}  = 1 } or 


{For  line  switched  network 


diCk)  >,  d.  8 n[Di(J0-di]  < £.k;  (3.3) 


For  message  switched  network 


di(kl  >-di  5 n[D.  (k)-di]/ti  < ®ik;}  (3. 


then  set  b.  1;  otherwise  set  b.  +•  0 
x 


- 62  - 


B.1.6 

n . *-  mx  . ; 
i i 

B.1.7 

V k s.t.  FiCk)  * READY  and  ni  > Z^k), 
set  FiCkl  «-  UP  and  t^Ckl  - NIL; 

B.1.8 

Vk  s.t.  FiCk]  * UP  and  R-Ck)  + SON, 
send  (ni,di,bi  ,i) ; 

B.  1.9 

CT  - 1 

B.2. 1 

T13  Condition  13: 

■ SON; 

B.2.2 

Vk  i i s.t.  F.(k)  - UP,  then  R^k)  ■ NIL; 

B.2.5 

MSG Cm,d  » »,b ,1)  or  FAlL(H); 

B.2.4 

CT  = 0 

B.2.5 

Fact  13: 

If  MSG,  then  min.. 

B.2.6 

Action  13: 

d^  * «; 

B.2. 7 

If  MSG,  then  n.  *■  m; 

B.2.8 

Y k s.t.  F (Jc)  - READY  and  ^ > Z^k), 
set  F.  Ck)  - UP  and  ^(k)  - NIL; 

B.2.9 

Vk  s.t.  F.  Ck)  =*  UP  and  R.  (k)  f SON, 

send  (ni,di,bi,i); 

B.2.10 

R. CHI  - NIL; 

B.2.11 

Cancel  the  flow  to  node  i and  modify  the 

routing  variables  by  setting  ■*  0 (for 

message  switching)  f^f  = 0 (for  line 

switching) ; 

B.2.12 

CT  = 1. 
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B.3.1 

Cl  Condition  1: 

R.CSO  * SON; 

B.3.2 

Z k * Z s . t . Ri  (k) 

» SON  and  F^k) 

* UP; 

B. 3.3. 

MSG  (m,d  = »,b,Jl) 

or  FAIL  Ci) ; 

B.3.4 

CT  = 0 

B.3.S 

Action  1: 

R.  Ci)  - NIL; 

B.3.6 

reroute  the  flow 

to  node  Z while 

arbitrarily 

redistributing  it  through  the  remaining  sons 
and  modify  the  routing  variables  correspondingly. 


STATE  S2 


B.4.1  T21 

Condition  21: 

yk  s.t.  F.(k)  = UP,  then  N^QO  * n^  ■ mx^ . 

B.4.2 

3 keAi  s.t.  DiCk)  id.; 

B.4.3 

If  CT  = 0,  then  MSG; 

B.4.4 

yk  s.t.  R.  (k)  = SON,  then  D^(k)  / <». 

B.4.5 

Fact  21: 

d.  f . 

l 

B.4.6 

Action  21: 

Rerouting; 

Caclculate  a=min  {D.(k}};  (3.4) 

k:keA. 

l 

B.4.7 

let  kQ  be  any  neighbor  that  achieves  the 

minimum  in  (3.4). 

B.4.8 

For  line  switched  network: 

B.4.8.1 

If  there  is  any  node  q s.t.  F^(q)“UP 

with  f.  >0,  then  for  all  neighbors 
lq 

keA,  do: 

l 

B.4.8. 2 

aik  = DiLkl  * a: 

B.4.8.3 

cancel  all  outgoing  links  corresponding 

to  incoming  links  that  have  been  cancelled 
by  fathers;  let  f'.^  be  the  remaining 
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B.4.S.4 

B.4.8.S 


B.4.8.6 

B.4.8.7 
B.4.9. 1 

B.4.9.2 
B.4.9. 3 
B.4.9. 4 


B.4.9. 5 


B.4.10 
B.4. 11 


Aik  * min{t'.k,naiki; 

SET  NEW  FLOW  (Vk  s.t.  F.  Ck)  » UP) 


,0 

<-new  _ / .f 


"ik 


f'  - A., 
ik  ik 


k t A. 


keAi,k^ko 


£'..  + l A.,+any  new  flow 
; 1K  keA.  lk 

k-ko 


if  f^k  = 0 7k  s.t.  F^(k)  * UP,  then 
any  new  flow  is  routed  through  kQ. 
For  message  switched  network: 

If  t.  > 0,  then  for  all  neighbors 
keA.  do: 


aik  = DiCk)  'a; 

Aik  = min{^ik)naik/t.}; 

SET  NEW  FLOW  (?k  s.t.  F^k)  « UP). 


new 

i . , 
ik 


0 

?ik 


"ik 


WA. 
keAi,k^k0 
k-k.  ; 


ik  . L ik 
keA^ 

k*k 


If  t.=0,  then  set  4-,  =1  and 

1 ik. 

0 

¥k  f kQ  s.t.  F.(k)  = UP,  set  4ik  “ 0 
Vk  s.t.  R ^ Ck)  = SON  send  (n.^  .cT  .b^  ,i) ; 

Vk  s.t.  F.  Ck)  * UP,  set  Rx(k)  - NIL; 


set  RiCkQ)  SON; 


..new 


^k  s.t.  k t k and  f.,  >0  (for  line 

O lk  ■ ■■■ 

new 

switched  network), 4. k > 0 (for  message 
switched  network),  set  R.  (k)  SON; 
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B.4. 12 

-k  s.t.  F^k)  = UP,  set  NiCk)  *-  NIL; 

B.4.13 

CT  «-  1 

8.5.1 

T22 

Condition  22 

Yk  s.t.  R^Ck)  a SON,  then  N^(k)  ■ nx^  > ni , 
DiCk]  i • and  F.Ck)  *=  UP; 

B.S.2 

CT  - 0 

B.S.3 

Action  22: 

Same  as  Action  12. 

B.6.  1 

T22 

Condition  22 

: Either  same  as  Condition  1 or 

Fail  (Z)  s.t.  Ri(Z)  i SON; 

B.6.2 

CT  = 0 

B.6.3 

Action  22: 

Same  as  Action  1 and  in  addition  set  CT  1 

B.7.1 

T23 

Condition  23 

: Same  as  Condition  13. 

B.  7.2 

Fact  23: 

Same  as  Fact  13. 

B.7.3 

Action  23: 

Same  as  Action  13. 

STATE  S3 

B.8.1 

T32 

Condition  32: 

~k  s.t.  F^k)  = UP,  mxi  * N^(k)  > n^, 

DiCk]  i 

B.  3.2 

Fact  32: 

d.  = «,  uk  s.t.  FiCk)  » UP,  then  R.Ck)=NlL. 

B.8.3 

Action  32 : 

Let  k*  be  a node  that  achieves 

min 

k:F.(k)=UP 

l'-  i 

B.8.4 

If  BiCk*)  = 1 .then  b.  1; 

B.8.5 

R.(k*)  «-  SON; 

B.S.6 

nA  «-  mxi; 

B.8.7 
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B.S.8 

T k s.t.  F.(k)  = READY  and  > Z.(l) 

set  F.(k)  •*-  UP  and  N.Qc)  - NIL. 

B.S.9 

Vk  s.t.  F.(kl  = UP  and  R.Ck)  + SON, 

B.8.10 

send  Cni,di,bi,i) ; 

For  line  switching:  any  new  flow  is  routed 

through  k* . 

For  message  switching:  set  1; 

B.8. 11 

CT  - 1 

STATE  S2 

B.9.1 

T22 

Condition  22 

Same  as  Step  B.5. 1 

B.9.2 

Action  22: 

Same  as  Action  12. 

B. 10. I 

T23 

Condition  23 

Same  as  Condition  13. 

B. 10.2 

Fact  23: 

Same  as  Fact  13. 

B.10.3 

Action  23: 

Same  as  Action  13. 

B.ll.l 

C2 

Condition  2: 

Same  as  Condition  1. 

3.11.2 

Action  2 : 

Same  as  Action  1. 

C.  Operation 

BfiBiassas: 

Done  by  the  Message  Processor  at  the  SINK 

C.l.  For  FAIL  (A) 

C.1.1  F.OD  *-  DOWN  ; CT=0:EXECUTE  FINITE- STATE-MACHINE  for  SINK. 

C.2  For  WAKE (&) 

(Fact:  = DOWN,  see  7 in  Section  3.3.2) 

C.2.1  wait  for  end  of  WAKE  synchronization  (see  10  in  Section  3.3.2); 

if  WAKE  synchronization  is  successfully  completed,  then 
C.2. 2 FSINKU)  - READY. 

C.2. 3 CT  - 0; 
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C. 3 For  MSGQn,d,b,a) 

c.3.1  NSiNKC*)-m; 

C.3.2.  CT  «-  0; 

C. 3. 3 EXECUTE  FINITE-STATE-MACHINE  for  SINK. 

D.  Finite-State-Machine  for  SINK 

■i:xsaa8ss3sss::s3i:::::s333s 

D.  1.1 
D.1.2 
D.  1.3 
D.1.4 

D.1.5 

0.1.6 


STATE  S2 


D.2.1 

T21 

Condition  21: 

Yk  s.t.  fsinkOO  = UP,  then  NgINKtk)  = nSINK; 

0.2.2 

MSG; 

D.2.3 

Action  21: 

Yk  s.t,  FSINKCkl  = UP,  set  NSINKCk)  NIL; 

D.  2.4 

CT  1; 

D.3.1 

T22 

Condition  22: 

{CT=0}  and  { FAI L or  WAKE } . 

D.3.2 

Action  22: 

Same  as  Action  12. 

r 


STATE  SI 

T12  Condition  12:  Either{CT-0>  and  {FAIL  or  WAKE}; 

or  the  SINK  decides  to  start  a new  iteration. 
Action  12:  If  FAIL  or  WAKE,  then  nSJNK  * nsiNK  * 1; 

Yk  s.t.  FSINKCk}  = READY  .set 

fsink^  UP  an<*  nsink^  NIL; 

Yk  s.t.  FSINKCk)  * UP,  send  (nsmy.,0 ,0 ,SINK) ; 

CT  1 . 
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3.4:  Properties  and  Validation  of  the  Protoco Is 

3.4.1:  Introduction 

Some  of  the  properties  of  the  protocols  have  already  been 
indicated  in  previous  sections.  We  now  turn  to  state  those  properties 
explicitly,  along  with  some  others  that  have  not  yet  been  shown. 

We  begin  with  some  definitions  and  notations,  then  we  state 
properties  that  hold  throughout  the  operation  of  the  network,  some  of 
them  referring  to  the  entire  netowrk  at  a given  instant  of  time  and 
some  to  a given  node  or  link  as  time  progresses.  Then  a series  of 
theorems  is  stated  which  enables  us  to  prove  the  recovery  of  the 
network  after  topological  changes.  Finally,  we  show  that  the  extended 
protocols  reduce  in  fact  to  the  basic  protocols  in  absence  of  topological 
changes. 


3.4.2:  Notations  and  Definitions 

In  this  subsection,  we  present  several  notations  and  definitions 
that  are  used  throughout  this  work.  The  notations  F^.Q:L,  R^Ckl, 
FAILO),  MSG(m,d,b),  n.,  d.,  N.(k),  D^k),  Z.OO,  SI,  S2,  S2,  S3,  Cl, 
C2,  PC  Cm)  and  others  have  been  already  introduced.  We  add  the  time 
in  parentheses  whenever  we  want  to  refer  to  the  above  quantities  at  a 
given  time  t;  for  instance,  R^OO(t),  n^t),  N^CJOCt),  etc.  We  also 
use  the  following  notations: 
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SX [n]  = state  SX  with  node  counter  number  n. 

s^(t)  » state  and  possibly  node  counter  number  n^  of  node  i at  time  t. 

Therefore  we  sometimes  write  s^(t)  = S3  for  instance  and  some- 
times s^(t)  = S3[n]. 

SON^  - set  of  nodes  {k:R^(k)  ■ SON}.  We  use  either  SON^  or 
{ k : R^OO  = SON}  at  our  convenience. 

T>2[t,i, (nl,n2]  means  transition  to  state  S2  (from  an  arbitrary  state) 
occurs  at  time  t at  node  i;  in  this  transition  node  i changes  its  node 
counter  number  from  nl  to  n2.  If  nl  is  arbitrary  we  write  p instead 
of  nl. 

T21[t,SINK, (nl.nl)]  means  proper  completion  of  an  iteration  with  counter 
number  nl. 

Routing  Graph 

At  a given  instant  t,  a Routing  Graph  RG(t)  is  defined  as  the 
directed  graph  whose  nodes  are  the  network  nodes  and  whose  arcs  are 
given  by  the  pointers  R^OO  = SON,  namely,  there  is  an  arc  from  node  i 
to  node  i in  the  Routing  Graph  RG(t)  if  and  only  if  R^ (A) (t)  = SON. 

(in  other  words  l e SON^(tll,  or  in  words  if  and  only  if  1 is  a son 
of  i at  time  t.  The  graph  RG(t)  has  some  very  important  properties  and 
for  describing  them,  a definition  of  an  order  for  the  states  is  needed. 
Therefore,  we  define  that  S3  > S2  ■ S2  > SI,  and  from  now  on,  we  agree 
that  Sx  i Sy  means  that  Sx  > Sy  or  Sx  = Sy.  We  also  define  the  terminating 
nodes  of  the  RG(t)  to  be  those  nodes  in  the  network  which  have  no  sons 
at  time  t.  For  instance,  Fig.  7b  is  the  Routing  Graph  of  the  network  in 
Fig.  7a.  Notice  that  the  SINK  is  always  a terminating  node. 
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For  conceptual  purposes,  we  regard  all  the  actions  associated  with 
a transition  or  a change  of  the  Finite-State-Machine  to  take  place  at  the 
time  of  the  transition. 

3.4.3:  Theorems 

Theorem  3.1:  Cloop-freedom) . 

At  an/  instant  of  time  t,  the  RG(t)  consists  of  a loop-free  directed 
pattern  (termed  lattice  from  now  on]  with  the  following  ordering 
properties: 

i]  the  terminating  nodes  of  the  lattice  are  the  SINK  and  all  nodes  in  S3. 

ii]  if  AeSON.Ct),  then  n„Ct)  * n.(t). 

1 JL 

iii]  if  AcSON^Ct]  and  n^Ct]  * n^t),  then  s^tt)  * si(t). 

The  proof  of  Theorem  3.1  is  given  in  Appendix  A.  According  to  the 
theorem,  the  Routing  Graph  has  at  any  instant  of  time  the  desirable 
loop-freedom  property.  It  should  be  noticed  here  that  isolated  nodes 
also  belong  to  the  Routing  Graph.  From  the  theorem  we  can  realize 
that  certain  ordering  in  the  Routing  Graph  is  maintained  by  the  protocols 
at  each  instant  of  time  throughout  the  operation  of  the  network.  The 
order  is  formed  by  concatenation  of  (n^,s^)  which  is  nondecreasing  when 
moving  from  the  peripheries  towards  the  terminating  nodes  of  the  pattern. 

Until  now,  properties  of  the  entire  network  at  each  instant  of  time 
throughout  the  operation  of  the  network,  have  been  stated.  In  the  next 
theorem  we  refer  to  local  properties  as  time  progresses. 
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Theorem  3 . 2 

i)  For  a given  node  i,  the  node  counter  number  n^  is  nondecreasing 
with  time  and  the  messages  MSGCm,d,b)  received  from  a given  neighbor 
have  nondecreasing  numbers  m. 

ii)  Between  two  successive  proper  completions  Cm)  and  PCCm),  for  each 

M 

given  m with  m s m s m,  each  node  sends  to  each  of  its  neighbors 
at  most  one  message  MSG(m,d,b)  with  d ? ». 

iii)  Between  two  successive  proper  completions  PC(m)  and  PC(m),  for 
each  given  m with  m s m s m , a node  enters  each  of  the  sets  of 
states  {Sl[m] ),{S2[m] ,S2[m] ),{S3[m]  } at  most  once. 

iv)  All  "Facts"  in  the  formal  description  of  the  algorithms  in  Section 
3.3.3  are  correct. 

A third  theorem  describes  the  situation  in  the  network  at  the  time 
proper  completion  occurs: 

Theorem  3,3 

At  PCCm),  the  following  hold  for  each  node  i: 

i)  If  n.  *=  m,  then  s.  = SI  or  s.  = S3. 

l i i 

ii)  If  a message  MSGCm.d.b)  with  d / ® is  on  its  way  to  i,  then  s^  = S3 
and  n^  = m. 

iii)  If  either  * m and  s^  = SI)  or  n^  < m,  then  for  all  k s.t. 

a UP  it  cannot  happen  that  (N^Ck)  = m,  D^(k)  + «}, 

A combined  proof  is  necessary  to  show  that  the  properties  appearing 
in  Theorems  3.1,  3.2,  3.3  hold.  The  proof  uses  a two-level  induction, 
first  assuming  properties  at  each  proper  completion  until  PCCm)  say, hold, 
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then  showing  that  the  other  properties  hold  until  the  next  proper 

a 

completion  named  PC(m)  and  finally  proving  that  the  necessary  properties 
hold  at  PC(S).  The  second  induction  level  proves  the  properties 
between  successive  proper  completions  by  assuming  that  the  property 
holds  until  just  before  the  current  time  t and  then  showing  that  any 
possible  event  at  time  t preserves  the  property.  The  entire  rigorous 
procedure  appear  in  Appendix  A. 

I 

In  order  to  introduce  properties  of  the  protocols  regarding  normal 
activity  and  recovery  of  the  network,  the  following  definitions  are 
necessary: 

Definition 

We  say  that  a link  (i,l)  is  potentially  working  if  ft  DOWN 

and  F^Ci)  / DOWN,  and  a link  (i,&)  is  working  if  F^(H)  = F^fi)  = UP. 

Two  nodes  in  the  network  are  said  to  be  potentially  connected  at  time 
t if  there  is  a sequence  of  links  that  are  potentially  working  at  time  t 
connecting  the  two  nodes.  A set  of  nodes  is  said  to  be  strongly 
connected  to  the  SINK  if  all  nodes  in  the  set  are  potentially  connected 
to  the  SINK  and  for  all  links  (i,£)  connecting  those  nodes, we  have  either 
F.U)  “ F^ti)  = UP  or  F.(fc)  * F^fi)  = DOWN. 

Definition 

Consider  a given  time  t,  and  let  ml  be  the  highest  counter  number  of 
iterations  started  before  t.  We  say  that  a pertinent  topological  change 
happens  at  time  t if  the  algorithm  at  a node  i with  n^Ct-)  * ml 
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receives  at  time  t a message  FAIL(Jl)  or  if  WAKE (J.)  is  received  at  i at 
time  t and  the  WAKE  synchronization  is  successful.  Observe  that  a 
pertinent  topological  change  happens  if  and  only  if  node  i has  a link 
(i,£)  such  that  at  time  t,  F^Ql}  changes  from  DOWN  to  READY  or  from  either 
UP  or  READY  to  DOWN. 

Theorem  3 . 4 

Let 

L(t)  = {nodes  potentially  connected  to  SINK  at  time  t); 

HCtJ  = {nodes  strongly  connected  to  SINK  at  time  t). 


Suppose 


Til 2 [ 1 1 , SINK,  (ml, ml)]  (3.12) 

namely  an  iteration  is  started  at  time  tl  with  a number  that 'was 
previously  used.  Suppose  also  that  no  pertinent  topological  changes  have 
happened  while  n^^  = ml  before  tl  and  no  such  changes  happen  ater  tl 
for  long  enough  time.  Then  there  exist  times  tO,  t2,  t3  with 
tO  < tl  < t2  < t3  < °°  such  that  a),  b) , c),  d)  hold: 


(a)  T21 [tO,  SINK,  (ml, ml)];  (5.13) 

(b)  -r/t  e [t0,t3]  , we  have  H(t)  = L(t)  = L(tO); 

(.c)  for  all  i e L(tO),  we  have 

Ti|i2[t2i,  i,  (ml, ml)];  15.11) 
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for  some  time  t2^  e [tl,t2]; 

C3.1S) 

d)  i)  T21[t3,SINK, (ml.ml)] ; 

ii)  RG(.t3)  for  all  nodes  in  L(.tQ)  is  a lattice  with  a single 
terminating  node  - the  SINK. 

In  words,  Theorem  3.4  dictates  that  under  the  given  conditions,  if 
a new  iteration  is  started  with  a number  that  was  previously  used,  then 
proper  completion  with  the  same  number  has  previously  occured  and  the  new 
iteration  will  be  properly  completed  in  finite  time  while  connecting  all 
nodes  of  interest  (namely,  those  in  L(tO))  to  the  SINK,  both  strongly  and 
routingwise.  The  proof  of  Theorem  3.4  appears  in  Appendix  B. 

The  recovery  properties  of  the  protocols  are  described  in  Theorems 
3.3  and  5.6.  The  proof  of  Theorem  3.5  appears  in  Appendix  B. 

Theorem  3 . 5 

Let  L(t),  H(t)  be  as  in  Theorem  3.4.  Suppose 

T^2[tl,  SINK,  (ml, m2)]  , m2  * ml  , (3.16) 

namely  an  iteration  is  started  at  time  tl  with,  a number  that  was  not 
previously  used.  Suppose  also  that  no  pertinent  topological  changes  happen 
for  a long  enough  period  after  tl.  Then 

a)  There  exists  a time  t2,  with  tl  s t2  < ”,  such  that 
i)  for  all  i e L(t2) 

T^2[t2^ , i,  (ip  ,m2)]  ; 


(3.17) 


happen  at  some  time  t2^  with  tl  £ t2^  £ t2; 

11)  HCt2)  = L C 1 2 ) 

b)  There  exists  a time  to  < 00  such  that 

i)  T21[t3,  SINK,  Cm2, m2)];  (3.18) 

li)  Yt  £ [t2,t3],  we  have  H(t)  = L(t)  = HCt2); 
lii)  RG(t3)  for  all  nodes  in  L(t3)  is  a lattice  with  a single 
terminating  node  - the  SINK. 

Part  (a)  of  Theorem  3.5  dictates  that  under  the  stated  conditions, 
all  nodes  in  L(t2)will  eventually  enter  state  S2[m2] . Part  b) 
dictates  that  the  iteration  will  be  properly  completed  and  each  node  potential 1\ 
connected  to  the  SINK  at  time  PC  (m2)  will  also  have  at  least  one  routing 
path  to  the  SINK. 

Finally,  we  observe  that  reattachment  of  a node  loosing  its  only  path 
to  the  SINK, or  leaving  state  S2,or  bringing  a link  up  requires  an 
iteration  with  a counter  number  higher  than  the  one  the  node  currently  has. 

In  the  next  chapter  we  present  a special  protocol  that  causes  such  an 
iteration  to  be  started  in  finite  time.  Here  only  the  following  assumption 
is  needed: 

Assumption  A , 

Suppose  that  a node  i with  n^Ct-)  = m detects  at  time  t a failure  of 
one  of  its  neighboring  links  or  succeeds  in  WAKE  synchronisation  with  its 
neighbor  l while  Z^C.Jl)  = m at  time  t,  then  the  SINK  either  has  started 
a new  iteration  with  counter  number  strictly  higher  than  m before  t,  or 


will  start  such  an  iteration  in  finite  time  after  t 
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Theorem  3.5  and  the  assumption  are  combined  in  the  following 

theorem: 

Theorem  3.6  (Recovery) 

Let  L(t),  H(.t)  be  as  in  Theorem  3.4.  Suppose  there  is  a time  tl 
after  which  no  pertinent  topological  changes  happen  in  the  network  for 
long  enough  time.  Then  there  exists  a time  t3  with  tl  £ t3  < such 

that  proper  completion  happens  at  t3  and  such  that  all  nodes  in  L(t3) 
are  on  a lattice  with  a single  terminating  node  - the  SINK,  and  are 
strongly  connected  to  the  SINK. 

Proof 

Let  tO  f tl  be  the  time  of  detection  of  the  last  pertinent  topological 
change  before  or  at  tl.  Let  node  i be  the  node  detecting  it  and  let 
m = n^C.tO-).  Then  by  assumption,  the  SINK  starts  a new  iteration  with 
counter  number  strictly  higher  than  m in  finite  time.  Let  t2  < “ 
be  the  time  the  SINK  starts  such  an  iteration  with  number  ml  > m. 

Since  by  the  definition  of  pertinent  topological  change,  m is  the  largest 
number  at  time  tO,  we  have  that  tO  < t2.  By  the  conditions  of  this 
theorem,  no  pertinent  topological  changes  happen  after  time  tO  for  a 
long  enough  period,  so  that  no  such  changes  happen  after  time  t2. 
Consequently  Theorem  3.5  holds  after  this  time  and  the  assertion  of  the 
Theorem  follows. 

Q.E.D. 

This  completes  the  proof  that  our  extended  protocols  possess  the 
required  properties  of  being  distributed,  loop-free  and  recoverable. 
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A final  theorem  is  needed  for  showing  that  they  reduce  to  the  basic 
protocols  otter  ail  topological  changes  have  occured 

Theorem  3 ~ (Optimality) 

Let  L(t)  be  as  m Theorem  34,  Suppose  there  is  a time  tl  after 
which  pertinent  topological  changes  never  happen  in  the  network  Let 
also  the  inputs  to  the  network  be  stationary  and  let  to  as  ,n  Theorem 
3.6.  Then  the  network  L(to)  will  be  brought  to  the  minimum  average 
delay  over  all  routing  assignments- 

Proof 

By  Theorem  3.6  there  exists  a time  t5  with  tl  s t3  ; « such  that 
proper  completion  happens  at  t3  and  such  that  ail  nodes  in  L(t5;  are  on 
a loop-free  lattice  terminated  only  at  SINK  After  time  t3  the  conditions 
of  Theorem  2.2  hold  and  the  algorithms  proceed  exactly  as  the  basic 
algorithms.  Therefore,  the  network  L(_t3j  will  be  brought  to  the  minimum 
average  delay  over  all  routing  assignments. 

q . e .d . 
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CHAPTER  4 


ADDITION  OF  A PROTOCOL  FOR  TRIGGERING  ITERATIONS 

4.1:  Introduction 

In  the  previous  chapter  we  have  described  two  distributed  routing 
protocols,  which  are  failsafe,  namely  the  protocols  operate  smoothly  under 
all  circumstances.  However,  to  show  their  ability  to  cope  with  topological 
changes,  an  assumption  has  been  made,  that  each  time  the  SINK  has  to 
start  a new  iteration  with  any  specified  number,  it  indeed  starts  it 

! 

and  does  it  in  finite  time.  The  specific  way  of  triggering  a new 
iteration  was  of  no  importance  from  our  point  of  view  as  long  as  the 
assumption  really  held. 

There  exist  several  procedures  for  starting  a new  update  iteration 
and  setting  the  corresponding  nSINK  in  a way  that  satisfy  the  above 
required  behavior  of  the  SINK.  A simple  procedure  is  that  at  given 
intervals  of  time,  or  as  a result  of  the  detection  of  a change  in  the 
traffic  patterns,  the  SINK  increments  and  starts  a new  update 

iteration.  This  procedure  may  make  use  of  a time-out  to  trigger  a new 
update  iteration  if  the  previous  one  is  not  properly  completed  within 
certain  time.  If  there  is  a topological  change  in  the  network  after 
proper  completion,  there  is  no  direct  triggering  of  a new  update  iteration, 
and  thus  recovery  can  be  achieved  only  whenever  the  SINK  decides  to  start 
a new  update  iteration.  In  addition,  this  procedure  unnecessarily 
increments  nsi;sJ)(,  for  every  update;  hence  an  unnecessarily  large  number  of 
bits  to  represent  n^^  is  required.  These  two  disadvantages  are  overcome 


by  the  protocol  presented  in  this  chapter.  This  specific  protocol,  when 
combined  with  the  protocols  described  in  Chapter  3,  enables  us  to  show 
that  whenever  need  arises, the  SINK  starts  a new  update  iteration  with  a 
specific  counter  number,  within  finite-time. 

In  the  following  description  we  first  describe  the  protocol 
informally,  then  we  combine  it  with  the  previous  protocols  and  formally 
describe  the  resulted  protocols.  Finally,  an  explicit  theorem  is  given 
that  shows  the  main  new  property  of  the  resulted  protocols 


4.2:  Informal  Protocol 

4.2.1:  Introduction 

We  have  observed  that  loosing  a neighbor  or  bringing  a link  up 
requires  an  iteration  with  a counter  number  higher  than  the  number  of  the 
node  sensing  the  change.  A procedure  is  therefore  needed  for  each  node 
that  senses  a topological  change  to  ask  the  SINK  to  start  a new  update 
iteration  with  a specified  number.  Since  all  our  protocols  are 
distributed  it  would  be  better  to  develop  a distributed  procedure  to 
achieve  the  desirable  goal.  Therefore,  the  following  protocol  is 
distributed  in  nature.  In  the  following  description  we  first  show  how  to 
ask  the  SINK  a special  request  and  then  how  to  forward  this  request  through 
the  network  until  it  arrives  at  the  SINK. 


f 

- SI  - 

. 2 2.  Request  Messages 

Any  node  discovering  a topological  change  by  either  detecting  a 
failure  or  sensing  that  a link  is  ready  to  come  up  generates  (in  addition 
*o  all  other  operations  described  in  Chapter  3)  a special  control 
message  - REQO^).  The  number  n^  contained  in  the  message  is  the  current 
node  counter  number  of  the  generating  node.  Since  after  a topological 
change,  the  node  usually  needs  a new  update  iteration,  with  a counter  number 
higher  than  its  current  number,  this  message  functions  as  a request  message 
intended  for  the  SINK.  Before  proceeding  to  explain  how  REQ  messages  are 
transmitted  through  the  network,  let  us  first  assume  that  such  a message 
is  received  by  the  SINK.  In  such  a case,  when  the  SINK  receives  REQ(m) 
it  immediately  starts  a new  update  iteration  with  counter  number  (m+1) , 
provided  that  such  an  iteration,  or  an  iteration  with  a number  higher 
than  im*l)  has  not  been  previously  started.  This  procedure  assures  that  if 
all  REQ  messages  generated  within  the  network  arrive  at  the  SINK  in  finite  ' 
time,  then  the  assumption  made  at  the  end  of  Chapter  3 indeed  holds. 

In  addition,  the  SINK  is  allowed  to  start  a new  iteration  while 
increasing  the  counter  number  at  any  time. 

We  now  turn  to  describe  how  REQ  messages  are  treated  and  sent  by  each 
node.  When  a message  REQ(m)  is  generated  by  a node  or  arrives  at  a node, 
it  is  put  in  the  regular  queue  and  preessed  on  FIFO  basis,  as  all  other 
control  messages.  When  the  nodal  message  processor  takes  such  a message 
and  starts  processing  it,  it  first  compares  its  node  counter  number  with 
the  number  contained  in  the  message.  If  it  finds  that  its  numbet  is  higher 
than  the  number  contained  in  the  message,  then  it  discards  the  request 
message,  since  it  is  clear  that  the  requested  iteration  or  even  an  iteration 
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with  a higher  counter  number  has  already  been  started.  Otherwise,  namely 
if  the  node  counter  number  is  equal  to  or  less  than  the  number  contained 
m the  message,  the  request  message  should  be  sent  forward  towards  the 
SINK.  Therefore,  the  protocol  dictates  that  in  such  a case  the  node  send 
this  message  to  a specific  neighboring  node,  called  preferred  son  for 
reasons  to  be  discussed.  As  much  each  node  i must  keep  one  more  memory 
location  denoted  by  p^  for  storage  of  the  identify  of  its  preferred  son. 

The  description  of  how  the  preferred  son  is  chosen  is  deferred  to  the 
next  section.  The  protocol  also  dictates  that  a node  that  hasn't  a 
preferred  son  (because  of  a failure)  discards  any  REQ  message  it  receives. 

4.2.3:  Selection  of  the  Preferred  Son 

REQ  messages  are  transmitted  through  the  network  along  a succession 
of  preferred  sons.  To  insure  their  arrival  at  the  SINK,  the  preferred  son 
must  be  well- chosen.  The  protocol  dictates  the  following  way  for  selecting 
the  preferred  son:  Each  time  a node  enters  state  SI  (clearly  this  may 
be  done  only  when  T21  is  performed),  it  chooses  kQ  (see  B.4.7  in  Section 
3.3.3)  to  be  its  preferred  son.  Remember  that  kQ  is  the  only  node  through 
which  we  permit  to  increase  the  flow  even  from  zero.  As  such,  it  is 
"preferred"  in  some  sense.  In  addition,  each  time  a node  enters  state  S2 
from  any  state  and  hasn't  a preferred  son,  it  chooses  arbitrarily  a node 
k s.t.  R^ Ck)  = SON,  to  be  its  preferred  son.  This  method  of  selecting  the 
preferred  son  guarantees  that  if  a REQ(m)  message  is  generated  in  the 
network,  and  sent  as  described  in  Section  4.2.2,  an  iteration  with  counter 
number  (m+1)  or  higher,  will  always  be  started  within  finite  time.  This 
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4.3:  Fornal  Description 


In  this  section  we  repeat  the  formal  description  of  the  two  algorithms 
of  Section  3.3.3  while  adding  the  protocol  we  have  just  described  in  the 
necessary  places.  To  save  space,  we  do  not  copy  Section  3.3.3  here  again, 
but  only  show  whore  the  present  protocol  must  be  added. 


4.3.1:  Notations 


Two  main  additional  notations  are  needed  for  the  following  description: 


p^  = preferred  son; 


REQ(m)  = request  message. 


4.3.2:  The  Algorithms 


Same  as  in  Section  3.3.3  with  the  following  additions: 


After  step  A. 1.3  add: 


A. 1.4  If  p^  i NIL,  then  send  REQ(n^)  to  p_ 


After  step  A. 2. 4 add: 


A.2.S  If  p^  + NIL,  then  send  REQ(n^)  to  p^. 


After  A. 3. 8 add: 


A. 4 For  REQCiiQ 

A. 4.1  If  p^  ft  NIL  and  n^  s m,  send  REQ(m)  to  p^; 
otherwise  discard  the  message. 


i 
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After  6.1.9 

i 

B.1.10  Choose  any  node  k s.t.  R^Ck]  * SON  and  set  p^  *-  k. 
After  B.2.12  add: 

B.2.13  If  p.  = l , then  set  p.  +•  NIL. 

’ ri 

After  B.3.6  add: 

B.3.7  If  p^  = l,  then  set  p.  NIL. 

After  B.4.13  add: 

B.4. 14  Set  p.  k . 

ri  o 

After  B.8.11  add: 

B. 8.11  Set  p.  «-  k* 

After  C. 3. 3 add: 

C.  4 For  REQQn) 

C.4.1  CT  * 0 

C. 4.2  EXECUTE  FINITE-STATE-MACHINE  for  SINK. 

Change  0.1.1  to: 

D. 1.1  T12  Condition  12:  {CT  > 0}  and  (FAIL  or  WAKE  or 

REQCm=nSINKl}; 

Change  0.3.1  to : 

T22  Condition  22:  {CT  ■ 0}  and  {FAIL  or  WAKE  or 
REQ(m-nSINK) }; 


0.3.1 
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4.4:  Properties  and  Validation  of  the  Protocols 

Clearly,  the  algorithms  that  are  described  in  Section  4.3.2  are 
exactly  the  same  as  the  algorithms  of  the  previous  chapter  with  the  simple 
addition  of  REQ:  Therefore,  the  present  protocols  possess  the  same 
properties,  and  all  the  theorems  that  are  stated  in  Chapter  3,  remain 
correct  here  too.  In  this  section  we  only  state  the  additional  properties 
the  protocols  have,  due  to  the  specific  protocol  that  has  been  added. 

At  first,  an  additional  property  of  the  Routing  Graph  is  stated: 

Theorem  4.1 

The  following  ordering  property  is  maintained  in  RG(t)  at  any  instant 
of  time  t: 

If  P^Ct)  t NIL  and  n^  (t)  » n^t]  and  s^  (t)  = s^t)  = SI,  then 
d Ct)  < d Ct). 

The  proof  of  Theorem  4.1  appears  in  Appendix  A.  According  to  the 
theorem,  in  addition  to  the  ordering  properties  in  the  Routing  Graph 
that  are  stated  in  Theorem  3.1,  it  has  also  the  following  ordering 
property.  For  nodes  in  state  SI  with  the  same  node  counter  number,  which 
are  the  nodes  that  have  properly  completed  the  update  and  reroute  in  a 
certain  iteration,  the  estimated  marginal  delays  to  the  SINK  are  strictly 
decreasing  along  the  concatenation  of  preferred  sons. 

The  next  theorem  comes  to  substitute  the  assumption  made  at  the  end 


of  Chapter  3. 
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Theorem  4. 2 

Suppose  that  a message  REQQnl)  is  generated  at  some  time  t at  some 
node  in  the  network.  Then  the  SINK  has  received  before  t a message 
REQQnl)  or  will  receive  such  a message  in  finite  time  after  t. 

The  proof  of  Theorem  4.2  appears  in  Appendix  B.  Before  going  any 
further  we  want  to  give  here  an  equivalent  definition  for  a pertinent 
topological  change,  in  connection  with  request  messages. 


Equivalent  Definition  (for  pertinent  topological  changes) 

A pertinent  topological  change  happens  at  time  t if  and  only  if  a 
message  REQQnl)  is  generated  at  time  t,  where  ml  is  the  largest  update 
counter  number  available  at  time  t in  the  network. 

It  is  easily  seen  that  this  definition  for  a pertinent  topological 
change  is  equivalent  to  the  definition  given  in  Chapter  3. 

Now,  it  is  also  clear  that  the  Recovery  Theorem  3.6  and  the 
Optimality  Theorem  3.7  hold  without  making  the  assumption  that  was  needed 
there,  since  Theorem  4.2  actually  assures  the  existence  of  the  necessary 
conditions . 
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CHAPTER  5 

SIMULATION  PROGRAM 

Actually,  there  is  no  need  for  a simulation  program  in  our  work 
since  we  validate  the  protocols  analytically.  However,  there  were 
several  problems  in  expressing  the  conditions  for  executing  the 
transitions  from  one  state  to  another  in  the  Finite-State  Machine 
To  overcome  these  problems,  a simulation  program  was  run.  This  program 
simulates  the  operations  done  by  individual  nodes  in  the  network.  The 
details  of  the  program  are  provided  in  Appendix  C. 

As  a result  of  the  simulation.  Section  3.3.3  was  written  and 
property  R7  (see  Appendix  A)  was  validated.  In  this  chapter  we  merely 
give  an  example  that  shows  the  necessity  of  the  simulation  program. 

Example 

Step  B.4.3  of  Section  5.3.3  was  written  at  first  as  follows: 

B.3.4  MSG;  (5.1) 

There  is  need  for  this  step  to  condition  transition  T21 , otherwise 
Condition  21  and  Condition  22  may  hold  at  the  same  time  (see  step  B.6.1 
in  Section  3.3.3).  However,  the  simulation  has  shown  that  (5.1) 
leads  to  deadlock  in  certain  circumstances.  Here  is  an  example: 

Let  a node  i be  in  state  S2  with  two  neighbors  kl  and  k2,  i.e. 

F^(kl)  = F^(k2)  = UP,  and  kl  is  its  only  son,  i.e.  R^(kl)  = SON, 

R^(k2)  = NIL.  Suppose  that  N.(k2)  = mx^  = ml  > n.  , D.(k2)  i 03  i.e. 
node  i has  already  received  a message  from  neighbor  k2.  Suppose  that 


88 


at  this  point  link  (i.Xllfails,  i.e.  a FAIL(kl)  is  received  at  i.  Then 
by  step  B.7.1  of  Section  3.3.3  node  i goes  to  S3.  By  step  B.8.1  c-f 
Section  3.3.3  it  also  performs  T32  and  goes  to  S2.  At  this  point, 
no  further  actions  are  taken  in  the  Finite-State  Machine,  particularly, 
T21  is  not  performed  because  of  (5.1);  however,  T21  must  be  performed 
at  this  point,  otherwise  there  is  danger  that  it  will  never  be 
performed,  i.e.  aeadlock.  To  overcome  this  situation,  step  B.3.4 
appears  a^  in  Section  3.3.3. 


CHAPTER  6 


DISCUSSION  AND  CONCLUSIONS 

This  thesis  presents  protocols  for  constructing  and  maintaining 
loop-free  routing  tables  in  a data-network,  when  arbitrary  failures 
and  additions  happen  in  the  network.  In  addition,  an  optimal 
routing  is  obtained  in  steady-state  in  the  sense  that  the  delay  is 
minimized.  Several  topics  involved  in  these  protocols  deserve  further 
discussion. 

The  iteration  counter  numbers 

Evidently,  the  iteration  counter  numbers  involved  in  our  protocols 
are  increasing  infinitely.  This  does  not  cause  analytic  problems, 
however,  it  makes  difficulties  in  structured  implementation.  Therefore 
other  versions  of  the  protocols,  in  which  the  iteration  counter  numbers 
will  be  drawn  from  a finite  alphabet,  must  be  considered.  Such  versions 
are  under  current  study. 

The  parameter  n 

In  [GALL  77]  and  [SEG  77a]  it  has  been  proved  that  the  basic 
protocols  converge  to  the  minimum  delay  in  stationary  conditions  only 
if  the  parameter  n,  involved  in  the  algorithms  for  each  node  in  the 
network,  is  chosen  to  be  very  small.  Certainly,  much  larger  n's  are 
to  be  used,  in  order  to  allow  some  dynamics  of  the  routing,  so  that 
slowly  changing  traffic  requirements  can  be  followed.  In  [POU  78]  it 
was  shown  (by  simulation)  that  large  n's  still  insure  convergence. 


It  is  interesting  to  mention  here  that  if  optimality  is  not  seeked 
and  if  we  allow  very  large  n's  (n  °°)  , then  our  extended  protocols 
reduce  to  the  protocol  of  [SEG  77c],  since  at  each  iteration  only  one 
son  can  be  chosen. 

The  request  protocol 

In  Chapter  4 we  described  a very  simple  protocol  for  triggering  new 
iterations  when  need  arises  (because  of  topological  changes) . Though 
the  protocol  is  simple,  the  proofs  of  its  correctness  are  very  complicated 
Furthermore,  we  couldn't  validate  the  protocol  unless  we  assumed  that 
the  d's  (marginal  delay  of  the  nodes)  are  non-negative  integers. 

Therefore,  other  protocols  which  are  simple  and  at  the  same  time  can  be 
easily  validated  must  be  considered. 

State  S2 

State  S2  of  the  Finite  State-Machine  was  introduced  in  order  to 
prevent  T21  from  happening  at  a node  that  is  in  state  S2  and  discovers  a 
failure  on  one  of  its  links  or  receives  MSG(d  = “)  from  a nonsingle  son. 
This  avoids  nodes  to  update  routes  based  upon  invalid  information.  In 
addition,  this  precludes  proper  completion  from  happening,  thus  enables  us 
to  validate  the  request  protocol  seperately  from  all  other  proofs. 

However,  it  is  easily  noticed  that  S2  is  artificial  and  actually 
unnecessary.  Step  B.4.2  in  Section  3.3.3  insures  that  T21  is 
prevented,  if  there  is  danger  that  the  nodes  will  update  routes  based 
upon  invalid  information. 


We  have  not  omitted  this  state  (S2)  since  we  feel  it  gives  better 
and  easier  understanding  of  the  operations  done  by  each  node,  and  it  does 
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This  appendix  is  organized  as  follows:  We  start  with  several 
notations  that  are  used  in  the  following  proofs,  then  we  proceed  with 
the  statements  of  a few  properties  that  follow  immediately  from  the 
formal  description  given  in  Sections  3.3.3  and  4.3.2.  Lemmas  A.l  - A. 5 
and  Theorem  A.l  contain  the  proofs  of  Theorems  3.1,  3.2,  3.3  and  4.1, 
together  with  some  other  properties  needed  in  the  proofs  themselves.  For 
simplicity,  we  use  in  the  appendices  the  word  "algorithms"  to  describe 
Sections  3.3.3  and  4.3.2. 


Notations 


In  addition  to  all  the  notations -we  have  already  introduced  , we 
use  a compact  notation  to  describe  changes  accompanying  a transition,  as 
follows: 


Txy[t,i,MSG(ml,dl,bl,U),  SEND  (m2  ,d2  ,b2 , £2) , (nl,n2),  (dl,d2), 
CS0N1.S0N2),  (pl,p2) , (mxl , mx21] 


CA.ll 


will  mean  that  transition  from  state  Sx  to  state  Sy  occurs  at  time  t 
at  node  i caused  by  receiving  MSG(jnl,dl ,bl]  from  neighbor  Al;  in  this 
transition  i sends  MSG(jn2 ,d2,b2)  to  neighbor  A2,  changes  its  node  counter 
number  n^  from  nl  to  n2,  its  estimated  marginal  delay  to  the  destination 
di  from  dl  to  d2,  its  set  S0Ni  of  sons  from  S0N1  to  S0N2,  its  preferred 
son  p^  from  pi  to  p2  and  the  largest  update  counter  number  received  up 
to  now  mx^  from  mxl  to  mx2. 
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Similarly, 

Txy  [ t , i , FAI L CU  ] , SEND  (m2 , d2 , b2 , 22 ) , (n  1 , n2 ) ,(dl , d2 ) , (SON  1 , SON 2 ) , 

Cpl.p2) , (mxl,mx2)]  (A. 2) 

denotes  the  same  actions  as  above,  except  that  they  are  caused  by  receiving 
FAIL  message  from  neighbor  21.  — 

Another  compact  notation  is  used  to  describe  changes  which  are  not 
accompanied  by  a transition  and  are  dnne  in  the  Finite-State-Machine, 
as  follows: 

Cx[t,i,MSG(m,d,b,2) , CSON1 ,SON2) , (pi ,p2) ] (A. 3) 

will  mean  that  a change  is  caused  by  receiving  a message  MSG(m,d,b)  from 
neighbor  2 ; in  this  change  the  set  SO^  of  sons  is  changed  from  S0N1  to 
S0N2  and  the  preferred  son  pi  is  changed  from  pi  to  p2. 

Similarly, 

Cx[t,i,FAIL(2) , (SONl,SON2},(pl,p2)]  (A. 4) 

means  the  same,  except  that  the  change  is  effected  by  receiving  FAIL 
message  from  neighbor  2. 

For  simplicity,  all  arguments  in  the  above  notations  that  are  of  no 
interest  in  a given  description  are  supressed,  and  if  for  example  nl  is 
arbitrary  then  G|/,n2)  is  written  instead  of  (nl,n2).  Similarly,  if  one 
of  the  states  is  arbitrary,  ^ will  replace  this  state. 

In  particular  observe  that 


means  that  proper  completion  of  the  iteration  occurs  at  time  t. 

For  Txy[t]  or  Cx[t]  we  also  use  the  following  notations: 

t-  ■ time  just  before  the  transition  or  the  change. 
t+  • time  just  after  the  transition  or  the  change. 

Also 

[t,i.MSG[a,d,b,A)]  (A. 7) 

is  used  to  denote  the  fact  that  a message  MSG(m,d,b)  is  received  at 
time  t at  node  i from  node  A,  whether  or  not  the  receipt  of  the  message 
causes  a transition  or  a change. 

Similarly 

[t.i.FAILCi)]  (A. S) 

is  used. 

Properties  of  the  Algorithms 

R1  Any  change  in  ni,  s^  or  sending  any  message  MSG(m,d,b)  can  happen 

only  while  node  i performs  a transition.  A change  in  SON.  can  happen 
only  while  node  i performs  a transition  Txy  or  a change  Cx. 


! 
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R2  Txy  [t  ,i,SEND(m»d,b) , (<|»,n2) , (ij> ,d2) , (ij/,mx2)  ] implies  d«d2  . 
If  d + « , then 

i)  Txy  » T21  or  Ti^2 

ii)  n2  * m = mx2 

If  d ■ »,  then 
iiil  Txy  ■ T<|»3 
iv)  n2  ■ m 
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R9  T32[t,i,(NIL,SON2)f(NIL,p2M»,d2)]  implies: 


(i)  p2  £ SON 2; 


(ii)  d2  > d (t). 


RIO  The  Finite-State-Machine  has  two  types  of  transitions.  The  first 
type  is  effected  directly  by  the  incoming  message,  while  the 
second  type  is  caused  by  the  situation  in  the  memory  of  the  node. 
Each  message  can  trigger  only  one  t.-ansition  of  the  first  type, 
and  this  transition  corns  always  before  transitions  of  the  second 
type.  This  is  controlled  by  the  variable  CT  in  Section  3.3.3. 
Transition T22,  T21  and  T32  are  of  the  second  type,  transitions 
T13,  T23,  T23,  T22  and  the  changes  Cl  and  C2  are  of  the  first 
type.  Transitions  T12  and  T22  belong  to  both  types. 


Rll  The  possible  changes  of  F^(t)  are  given  in  Fig.  8.  The  types 


READY 


FAIL 

WAKE 


FiS-8  Possible  changes  of  F^(l). 

of  messages  causing  them  are  also  shown.  A pertinent  topological 
change  happens  if  F^(Jl)  -►  DOWN  or  F^(t)  changes  from  DOWN  to  READY 
at  a node  i with  n^(t-)  = ml,  where  ml  is  the  highest  counter  number 
of  iterations  started  before  t. 


The  following  lemma  proves  statement  i)  of  Theorem  3.2  and  shows 
the  role  of  the  node  counter  number  n. . Here  we  see  for  the  first  time 

l 

that  several  properties  have  to  be  proved  in  a common  induction. 
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Lemma  A. 1 

a)  Let 

[tl,i,MSG(jnl,dl,bl,A)] , 

[t2,i,MSG(m2,d2,b2,£)] , 
then  t2  > tl  implies  m2  i ml. 

b)  For  a node  i,  n^  is  nondecreasing  with  time. 

c)  Let  Mjtt.A)  denote  the  counter  number  m of  the  last  message 
MSG(m,d,b)  received  at  node  i before  or  at  time  t from  node  i. 
Then 

€ MiCt,A)  V*  e SON^t) 

Proof 

The  proof  proceeds  by  induction.  We  assume  a),  b) , c)  hold  up  to 
time  t-  for  all  nodes  in  the  network.  We  then  prove  that  any  possible 
event  at  time  t preserves  the  properties.  This  combined  with  the  fact 
that  a),  b),  c)  hold  trivially  at  the  time  any  node  comes  up  for  the 
first  time  completes  the  proof. 

a)  Suppose  t ■ t2. 

Then  by  FIFO  and  property  R2  it  is  clear  that: 

3 t3  s.t.  Txy[t3,A,SEND(ml,dl,bl)]  * nz(t3)  = ml 
9 t4  s.t.  Ta6[t4,A,SEND(m2,d2,b2)]  * nz(t4)  = m2 
with  t3  < t4  < t. 

By  induction  hypothesis  on  b)  nfc  was  nondecreasing  up  to  (but  not 
including)  time  t,  so  ml  s m2. 

q- 


(A.  9) 
CA. 10) 


(A. 11) 


(A.  12) 
(A.  13) 


e .d . 


J 
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b)  Here  we  check  all  possible  events  at  time  t. 

If  C*[t,i.tnl,n2)]  or  T21[t ,i,  (nl  ,n2)]  or  T22[t,i,  (jil,n2)] 
happens,  then  the  node  counter  number  n^  is  not  changed  so 
nl  ■ n2  (see  Action  1,  Action  2,  Action  21,  Action  22  in 
Section  3.3,3],  q.e.d. 

If  T32[t,i,(nl,n2)]  or  T22[t,i. Cn2,n2]]  or  T22 [t ,i , (nl ,n2] ] 
happens,  then  by  property  R3,  n2  > nl,  q.e.d. 

If  T12[t,i,  (nl,n2)  j(S0Nl,S0N2]]  happens,  then  by  induction 
hypothesis  on  c] 

nl  * MiCt-,k]  V k e S0N1. 

Since  in  T12  we  have  S0N1  * SON2,  then 
nl  $ MiCt-,k]  Vk  e S0N2 . 

By  applying  a]  at  time  t we  get: 

MiCt-,k)  s MiCt+,k]  =*  n2  ^k  e S0N2 
where  the  last  equality  follows  from  steps  B.l.l.B.1.6  in  Section  3.3.5 
Hence  nl  s n2,  q.e.d. 

If  T<|>3[t,i,  (nl,n2] , (S0N1  ,S0N2]]  happens,  then  the  transition 
might  be  caused  by  either  FAILOO  or  MSG(m,d,b,£) . 

If  FAIL 00,  then  n2  = nl  Csee  step  B.2.7  in  Section 
3.3.3],  q.e.d. 

If  MSGQn.d.b, Z]  then  from  steps  B.2.1,  B.2.7  in  Section 
3.3.3  we  know  that  Z e S0N1  and  n2  ■ m,  therefore: 
nl  « MiCt-,Al  s m = n2 

where  the  inequalities  follow  respectively  from 
induction  hypothesis  on  c)  and  by  applying  a)  at  time 


t. 


q.e.d. 


4 
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c)  Here  we  check  again  all  possible  events  at  time  t. 

If  Q/[t,i]  or  T22[t,i]  or  a received  message  causes  no 
transition,  then  from  Section  3.3.3  we  have: 

niCt+)  = niCt-l  and  SON^t-)  2 SON.(t+) 

From  induction  hypothesis  on  cl 

n^t-1  s MiCt-,k),  Vk  e SON.(t-). 

Therefore: 

niCt+l  s MiCt-,k),  Vk  e SONiCt+). 

Finally,  by  applying  a)  at  time  t we  get: 

niCt+l  < NLCt-.k)  $ M^(.t+,k],  Vk  e SON^Ct*),  q.e.d. 

If  T<|>3[t,i]  happens,  then  SON^(t+)  * NIL,  so  nothing 
has  to  be  proved. 

- If  T21ft,iJ  happens,  then  the  counter  number  of  the  last  message 
received  before  time  t from  any  neighbor  is  (see  step  B.4.1 
in  Section  3.3.3) 

niCt-)  » niCt+l  = mxiCt-) 
therefore, 

niCt+]  - Mi(t+,kl,  Vk  e S0N.(t+l,  q,e.d. 

If  T^2[t,i]  happens,  then  Csee  B.1.1,  B.5.1,  B.8.1,  B.9.1 
in  Section  3.3.3) 

niCt+)  » mxi Ct-)  - NiCk)Ct-)  - NTCt+.k),  Vk  e SONi(t+).  q.e.d. 

! 

. 


The  next  lemma  shows  what  are  the  messages  that  can  travel  on  a link 
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Lemma  A. 2 

a)  If 

[tl,i,MSG(jnl,dl,bl,A)], 

[t2,i,MSG(m2,d2,b2,A)] 
where  t2  > tl,  dl  ■ «,  then  m2  > ml. 

b)  If 

' [tl,i,FAILC«], 

[t2,i,MSGCm2,d2,b2,A)] 

where  t2  > tl,  then  m2  > n^Ctl)  and  also  m2  > n^Ctl). 


Proof 

a)  3t3  < tl  such  that 
W[t3,A,SEND(jnl,dl— ,bl,i) , 0,n2)] 

and  from  property  R2  we  have  ml  * n2. 

The  next  transition  of  node  l l must  be: 

T32[t4,A,Qi2,n3)] 

with  n3  > n2,  so  that  by  Lemma  A.l  b)  which  says  that  n£  is 
nondecreasing,  we  see  that  A will  never  send  any  message 
MSG(jn,d,b)  with  m « ml  after  t3.  R2  and  FIFO  at  node  i completes 
the  proof,  q.e.d 

b)  After  a failure,  a link  Cij*)  can  be  brought  up  only  with 

numbers  strictly  higher  than  Z .(A)  » max(n. ,n,)  . Since  n.  and 
n^  are  nondecreasing  numbers  by  Lemma  A.l  b) , the  proof  is 
completed,  q.e.d 


(A. 14) 

CA. IS) 


(A. 16) 
CA. 17) 


CA.18) 


(A.  19) 
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Lemma  A. 5 

If 

Tv2[tl,i,  GKflO]  CA-20:) 

then-yt  * tl  we  have  for  all  k s.t.  F^(k)(.t)  * READY  that 
ZiCk)Ct)  5 m.  Therefore,  no  link  is  brought  up  by  node  i with  number 
m after  entering  S2[m]  (.brought  up  means  F^k)  «■  UP}. 

Proof 


If  at  time  tl-  we  have  F.QOttl-)  - READY  and  Z.(k)(tl-)  < m,  then 

X 

link  Ci.k}  is  brought  up  by  node  i (F^(k)  UP)  at  time  ti(see  B.1.7, 

B.2.8,  B.5.4,  B.9.8,  B.9.2  in  Section  3.3.3). 

If  at  time  tl-  we  have  F.(k)Ctl-)  - READY  and  Z.(k)(tl-)  5 m,  then 
nothing  would  happen  at  time  tl  and  for  all  t > tl  Z^Qf.l(tl  m,  since 
Z^Qc)  is  nondecreasing  Cby  Lemma  A.l  b)). 

If  Fi(k)  has  been  set  to  READY  after  tl,  then  by  Lemma  A.l  b) 
ni(t)  5 m A/t  > tl 

and  clearly  Z.(k)(t)  z m yt  > tl,  q.e.d. 

Lemma  A. 4 

If  F,  (A) Ct)  - READY  and 


[t,i,MSG(m,d,b,A)] , 


(A. 21) 


then  m > Z-^Ht).  Observe  that  this  is  the  Fact  in  step  A.  2 in 


J 


Section  3.3.3. 
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Proof 

From  steps  A.l,  A. 2,  A. 3 in  Section  3.3.3  and  property  7 in  Section 
3.3.2,  F^OO  can  go  the  READY  only  from  DOWN  and  only  when  successful 
synchronization  of  WAKE(£)  occurs  at  i.  Let  tl  < t be  the  last  time 
it  occurs.  By  property  7 in  Section  3.3.2,  at  time  tl  there  are  no 
outstanding  messages  on  (i,£)  or  C^.i}  and  2^00  is  established  as 
max{n.,n„}  (see  A. 2. 2 in  Section  3i3.3).  Therefore,  the  message  in  (A. 21) 
must  have  been  sent  at  time  t2  > tl  and  since  node  i sends  messages  only 
to  nodes  k for  which  F^OO  ■ UP,  it  follows  that  F^(a)Ct2+)  ■ UP. 

But  F^(.£)  could  have  been  set  to  UP  only  from  READY  because  of  B.1.7, 

B.2.8,  B.5.3,  B.7.3,  B.S.8,  B.9.2  or  B.10.3  in  Section  3.3.3  and  not 
because  of  A. 3.1  and  in  all  the  above  cases  we  have  n^  > Z^Ci)  ■ Z^(Jl). 
Since  n^  is  nondecreasing  and  l sends  MSG(m,d,b)  only  with  m = n^, 
the  assertion  follows,  q.e.d 

Lemma  A. 5 

All  "Facts"  in  Section  3.3.3  are  correct. 


Proof 

The  Fact  appearing  in  step  A. 2 in  Section  3.3.3  is  proved  in 
Lemma  A. 4.  The  Fact  in  A. 3.1  follows  from  property  7 in  Section  3.3.2. 

Fact  32  is  correct  since  from  B.2.2,  B.2.6,  B.2.10,  B.7.1,  B.7.3,  B.10.1, 
B.10.3  in  Section  3.3.3  we  conclude  that 

T^3[i, (dl,d2) , (S0N1 ,S0N2)]  (A. 22 


implies  d2  ■ •»,  S0N2  » NIL. 
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Facts  13,12,23  and  23  follow  from  Lemmas  A.l  a)  and  A.l  c) , since  if 
MSG(m,d,b)  is  received  at  node  i at  time  t from  node  i and  T^3  or  T12 
happen,  then  i e SON^t-)  and 

I 

' 

m = number  received  at  time  t by  node  i from  £ % M^(t-,t)  % n^(t-). 


Fact  21  is  correct,  since  if 
T*2[i,(dl,d2)] 

happens,  then  d2  f • and  since  SON^  * NIL  iff  s.^  » S3,  q.e.d. 

The  next  Theorem  completes  the  proofs  of  Theorems  3.1,  3.2,  3.3 
and  4.1. 

Theorem  A. 1 

Let  PC  (in]  and  PC(m)  be  the  instants  of  occurence  of  two  successive 
proper  completions.  Then, 

a]  Theorem  3.3. 

b)  Consider  any  number  ml  £ m.  Let  m be  the  highest  counter  number 
m S ml  such  that  PC(m)  occurs.  Let  LPC(m,ml)  be  the  time  of 
occurrence  of  the  last  PC(m)  such  that  PC(m)  £ PC(m).  If  for 
any  i,k,  t £ PC(m),  we  have  either 

N.OO(t)  - ml  - m,  Di(k)(t)  i -,  Sj.tt)  i S3,  n^t)  - m (A. 23] 


or 


Na (k) Ct)  * ml  > m 


(A. 24) 

(A. 25) 
(A. 26) 
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with  dl  - D.CkHt)  - D'ik(j2). 

(Note:  In  words,  the  above  insures  that  a message  MSG(ml ,dl,bl)  was 
sent  and  received  no  earlier  than  LPCCm.ml)] . 

c)  Consider,  any  number  ml  $ m.  Let  m be  the  highest  counter  number 
m $ ml  such  that  PC  Cm)  occurs.  Let  LPCCm.ml)  be  the  time  of 
occurrence  of  the  last  PC  Cm)  such  that  PC  Cm)  S PC  Cm).  Then 

i)  [tl.i.MSGOnl.dl.bl,*)],  _ (A. 27) 

[t2,i,MSGCm2,d2,b2,Jl)] , d2  t ~ CA.28) 

where  LPCCm.ml)  { tl  < t2  s PC  Cm)  imply  m2  > ml 

ii)  If 

T21[tl,i, Cnl.nl)],  nl  * ml  (A. 29) 

[t2 ,i,MSG(m,d,b, A)]  , d + « (A. 30) 

where  LPCCm.ml)  S tl  < t2  s PC(m), 
then  m > nl. 

iii)  A node  i enters,  between  LPCCm.ml)  and  PCCm),  each  of  the 
following  sets  of  states  at  most  once: 

(Sl[ml] ),{S2[ml] ,S2[ml] ),{S3[ml] ). 

(Note:  Observe  that  for  the  particular  case  where  m ■ m this 
is  Theorem  3.2  iii)) . 

d)  i)  The  possible  transitions  or  changes  at  a node  are  the  following  , 

where  n2  * nl  and  n3  > nl:  T12 [ (nl ,n2) ] , T13[ (pi ,n2)] , 

Cl[Cnl,nl)] , T21[ Cnl.nl)] , T22[(nl,nl)] , T23[(nl,n2)] , 
T22[(nl,n3)],  T22[(nl,n3)] , T23[ (nl ,n2)] , C2[Cnl,nl)], 

T32[(nl,n3)]  and  C3[ Cnl.nl)]. 
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lij  T21[t,i,  Cnl.nl)]  implies  thatVk  s.t.  i e SON^Ct)  then 
sk(t)  = Sl[nl] . 

Theorem  3.1  and  Theorem  4.1. 

i)  Suppose 

T21[t,i, Cnl.nl)] 

happens  with  nl  ■ m , and  let  rl  be  the  last  time  before 
t such  that 

Tt/*2[Tl,i,  Oli,  nl)] 

happens.  Then  we  have  F^QOCrl)  = UP  if  and  only  if 
F-OcKt]  = UP  Vt  £ [tl.t]. 

ii)  If  for  some  t e ( PC (m) , PC  Cm) ] we  have 

T^2[t>i,0l'»n2)] , n2  « m 

Then 

3*1  e (t.PCCm)J  such  that  T21[Tl,i, Cn2,n21]  happens, 
and  $r2  e [t.PCCmi]  such  that  T23[t2,i] 
or  T22[r2,i]  happen 

If  3i,k,  t e C?C0n),PC(pi)]  such  that  for  some  t e (PCCm).t) 
holds 

[r.k.SENDCJ.d.i)]  , di<« 

and  if  node  i either  has  not  received  this  message  by  time  t, 
or  has  N^(k)Ct)  « m,  D^Ck)C£)  A 00  , then  -3tl  e [t,PC(m)]  such 
that 

s^(.tl)  3 S2[m]  or  s^Ctl)  * S3[m]. 


(A. 31) 

tA. 32) 

CA. 33) 

(A. 34) 

(A- 35) 

CA- 36) 
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Proof 


a)  As  said  before,  the  proof  proceeds  using  a two- level  induction. 

We  first  notice  that  a)  trivially  holds  at  the  time  the  network 
comes  up  for  the  first  time  (this  time  might  be  denoted  by  PC(0)). 
Then  we  assume  that  a)-g)  hold  at  every  time  up  to  and  including 
PC  (in) . Next  we  prove  that  b)-g)  hold  until  the  next  proper 
completion  PC(m),  using  the  second  level  induction.  Finally, 
we  show  that  a)  holds  at  PC(m),  thus  completing  the  proof. 


b)  Clearly,  a message  MSG(m,d,b)  with  m * N^(Jc)0t),  must  have  been 
sent  before  t.  We  have  to  show  that  such  a message  must  have 
been  sent  after  LPC(m,ml). 


For  the  case  (A. 24)  where  N^(k)(t)  > m,  suppose  the  message  has 
been  sent  before  LPCOn.ml),  then  it  implies  by  R2  that  at  LPCCm.ml) 
we  have  n^  > m contradicting  R8  and  implying  such  a message 
has  been  sent  and  therefore  received  after  LPC(m,ml). 


For  the  other  case  (A. 23)  where  N^(k)Ct)  * m,  D^CklCt)  t ®, 
s^Ct)  i S3,  n^(t)  • m,  assume  that  the  message  MSG(m,d,b)  has  been 
sent  by  k to  i before  LPC(m,ml)  and  no  such  a message  has  been 
sent  by  k to  i afterwards.  First  assume  the  message  is  on  its  way 
to  node  i at  LPCCm.ml).  This  implies  by  the  induction  hypothesis 
on  a)  ii)  applied  at  LPC(m,ml)  that  we  have  s^CLPCOn.ml))  = S3 
with  ^ » m.  However,  at  time  t,  s^Ct)  i S3  and  when  a node  leaves 
state  S3  it  strictly  increases  its  node  counter  number,  but  we 
have  n^Ct)  = m,  contradicting  the  assumption.  Next  assume  the 
message  has  arrived  at  node  i before  LPC(m,ml).  Since  the 
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situation  N^Ck)  * m,  D^Ckj  t 30  holds  until  time  t it  also  holds 
at  LPCOn.ml)  At  LPC(jn,ml)  s m.  If  nj  ■ m at  LPC(m,ml) 
then  by  the  induction  hypothesis  on  a)  i)  applied  at  LPC(m,ml) 
we  have  s^  * SI  Cwe  have  already  seen  that  CJ-PC (pa , m 1 i ) ? S3.  In 

either  cases, (n^  * m § s^  » Slior^n^  < m}by  the  induction  hypothesis 
on  a)  iii),  for  all  k s.t.  F^Ckl  ■ UP  it  cannot  happen  that 
(IT  (JO  * m,  Ck J + «}  at  LPC(m,ml),  asserting  a contradiction. 
Therefore,  (A. 25)  is  asserted.  q.e.d. 

cj  Suppose  c)i)ii)  and  iii)  are  true  for  all  nodes  in  the  network 
up  to  time  t-.  We  prove  c)  i)  and  c)  ii)  for  t2  »t  and  then 
prove  c)  iii)  for  t. 

i)  If  dl  * «,  then  m2  > ml  by  Lemma  A .1  a).  From  Lemma  A.i  a) 

m2  i ml.  So,  assume  dl  + 00  and  m2  * ml  and  we  are  going 

to  show  that  this  assumption  asserts  a contrdiction . 

If  dl  * » and  m2  = ml,  then  Lemma  A. 2 a)  and  Lemma  A.I  a) 
respectively,  imply  that  jjt3  t (tl,t2)  such  that 

[t3,i,MSG(jn,d— ,b,i}]  (A. 37) 

or  such  that 

[t3,i,MSG(m3,d3,b3,i,)]  (A. 38) 

with  m3  + m2  ■ ml.  Therefore  the  two  messages  received  at 
tl  and  t * t2  can  be  taken  as  consecutive.  So  using  b), 

FIFO  and  property  R1  it  turns  that  3t4  e [LPCOn.ml)  ,tl]  and 

3tS  e Ct4,t2)  such  that 

Txy[t4,i.,SEND (ml ,dl ,bl , i)]  , dl  t « ; 


TaB[t5,l,SEND[ml,d2,b2,i)],  d2  + •». 


(A. 33) 
(A. 40) 
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By  R2,  Txy  = T21  or  T^2  and  same  for  Tag.  But  by  induction 
hypothesis  on  c)  lii),  node  l cannot  enter  {S2[ml] ,S2[ml] } 
twice  between  LPC(m,ml)  and  PC(m),  so  that  the  only 
possibilities  are: 

{T^2[t4,t] } AND  {T2lItS,*]} 

and  no  other  transition  happens  between  t4  and  tS.  However,  in 
Ti|»2[t4,t],  node  l sends  a message  to  every  neighbor  except  sons, 
i.e.  except  those  nodes  that  belong  to  S0NiCt4-t-)  (see  steps 
B.1.8,  B.5.3,  B.8.9,  B.9.2  in  Section  3.3.3),  and  in 
T21[tS,t],  only  to  sons,  i.e.  to  nodes  that  belong  to  SON^(tS-) 
Csee  B.4.1Q  in  Section  3.3.3).  Since  no  other  transition 
happens  between  t4  and  tS  we  have  S0fT(.t4+)  * S0N^(t5-), 
contradicting  (.A.  39),  (A. 40). 

So,  m2  > ml  q.e.d. 

ii)  Clearly,  F^C£)C.t2-)  3 UP.  If  F^(Jl)(tl)  f UP,  then  Lemma  A. 4 
together  with  the  facts  that  ni  is  nondecreasing  (by  Lemma 
A.  lb)  and  that  Z^t)  is  established  as  in  step  A. 2. 2 in  Section 
3.3.3^  show  that  the  first  message  MSG(m2 ,d,b,i)  that  can 
be  received  by  node  i from  node  l after  tl  must  have  m2  > ml  = nl. 
Then  the  assertion  follows  from  Lemma  A.l  a). 

Suppose  now  that  F^(A)(tl-)  3 UP,  then  step  B.4.1  in  Section 
3.3.3  requires 

N^(£)(tl-)  * nl  ■ ml 

and  by  the  definition  of  LPC(m,ml)  we  have  nl  = ml  i m.  We 
now  distinguish  between  two  cases: 

If  D^(Z)(.tl-)  3 ",  then  3t3  < tl  (possibly  t3  < LPC(m,ml)) 


such  that 
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[t3,i,MSGCnl,dl-“,b,*)]  (A. 41) 

and  the  assertion  follows  from  Lemma  A. 2.  a). 

If  D UlC.tl-1  f »,  then  from  b)  it  follows  that 
3 t3  e [LPC(m,ml) ,tl)  such  that 

[t3,i,MSG(nl,dl,b,i)J  , dl  i ® CA.42) 

and  the  assertion  then  follows  from  c)  i). 

iii)  From  Lemma  A.l  b) , nj^  is  nondecreasing,  so  that  once  n^  is  increased, 
it  cannot  be  returned  to  the  old  value. 

From  Section  3.3.3,  a node  can  leave  {S2[ml] ,S2[ml] } only  via  T21 
or  T23  or  T23  without  changing  the  node  counter  number.  If  T23  or 
T23  happens  then  R3  shows  that  node  x will  strictly  increase  n^ 
when  leaving  {S3[ml]>.  If  T21  [(ml.ml)]  happens  then  c)  ii)  shows 
that  it  cannot  subsequently  receive  a message  with  d ^ 00  with  the 
same  ml,  and  in  order  to  enter  S2[ml]  again,  such  a message  should 
be  received.  Therefore,  a node  can  enter  { S2 [ml] ,S2[ml] } at  most 
once  between  LPC(m,ml)  and  PCCm). 

To  Sl[ml]  a node  enters  only  from  S2[ml],  so  that  it  cannot  enter 
Sl[ml]  twice  between  LPC(jn,ml)  and  PC(jii). 

If  a node  enters S3[ml] , by  R3  it  leaves  S3  only  with  a higher  n^, 
so  that  it  cannot  come  back  with  the  same  n^.  q.e.d. 

d)  l)  The  assertion  follows  immediately  from  Section  3.3.3  and  from  the  fact 
that  the  node  counter  number  is  nondecreasing,  stated  by  Lemma  A.l  b) 
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ii)  Recall  that  we  are  always  considering  times  before  PC(m). 

We  are  going  to  proved  ) iil  for  one  node  k such  that 
i c SON^Ct)  and  the  proof  for  all  other  fathers  of  node  i 
follows  in  the  same  way. 

Observe  that 

T21[t,i,(nl,nll]  (A. 43) 

implies  that  NjC.JtJCt)  = nl  for  all  t s.t.  F^l  = UP.  Note 
also  that  here  i e SON^(t)  implies  F^(k)  = UP,  so  that 
Ni(k)(t-)  * nl.  Note  further  that  D^CkKt-)  i »,  since  other- 
wise k was  sometime  before  t in  S3[nl]  and  could  attach  to  node 
1 only  if  i sent  to  k a message  with  counter  number  strictly 
higher  than  nl,  contradicting  T21 [t , i , (nl ,nl) ] . 

However,  N^Ck)(t-)  * nl,  D OOCt-)  / 00  implies  by  b)  that 
e [LPC(m,nl) ,t]  such  that 

Txy[r,k.SEND(nl,d,b,il]  , d f ® CA.44)' 

Now,  there  are  two  possibilities: 

If  i i SON^Cj-),  then  Txy  = Tij/2,  but  in  order  that  i e SON^(t), 
k must  have  performed  T21[tl,k]  at  some  time  tl  e O.t). 

On  the  other  hand,  if  i e S0N^(_t-},  then  Txy  = T21.  Therefore, 
in  either  cases,  k performed: 

T21[n,k,(nl,nl],(SONl,SON21]  , i c S0N2  CA.4S) 

at  some  time  n e [LPCCm.nl) ,t] . So  skCn+)  = SI [nl]  and 
i t SON^Cn*)- 

From  c)  and  the  fact  that  i e SON^ft),  one  can  easily  see  that 
k remains  in  SI [ml]  at  least  until  time  t, 

L 


q.e.d. 


Part  i)  of  Theorme  3.1  is  trivially  proved,  since  at  any  time  only 
the  SINK  and  nodes  in  state  S3  have  no  sons. 

Part  ii)  and  iii)  of  Theorme  3.1,  the  loop-freedom  property  and 
Theorem  4.1  are  proved  by  induction,  assuming  they  hold  up  to  time 
t-  (_t  £ PC (m) ) and  showing  that  for  any  possible  event  at  time  t,  these 
properties  are  preserved.  To  simplify  the  proof  we  look  at  the 
concatenation  (n.,s.)  and  write  Or»s.)  ^ Ov,s.)  if  n.  z n.  and 

XI  XX  KK  IK 

if  ni  ■ n^  implies  s^  i s^.  Using  this  notation  observe  from  d)  i) 
that 

Txy[t,i,  (jil,n2)]  (A.  46) 

implies  Cn2,y)  i (nl,x)  for  any  x and  y except  for  Txy  = T21. 

Note  further  that  the  induction  hypothesis  on  3.1  ii)  and  3.1  iii)  is: 
If  Jl  e SON^t)  then  (n^s^Ct)  * (ni,si)(.t)  Vt  « t-. 

Finally  notice  that  the  changes  of  interest  here  are  in  n^,s^,S0N^ ,p^ 
and  d^. 

We  now  turn  to  consider  all  possible  events  at  time  t: 

C<Ji[t,i]  ; only  S0Ni  is  changed.  Since  SQN^(t)  2S0N^Ct+)  the 
properties  trivially  hold  at  time  t+. 

T22[t,i];  only  s^  and  possibly  SONi  are  changed.  Since  s^Ct-t-)  = s^(t-) 
and  SON^Ct-)  2SON^Ct+l  the  properties  are  preserved. 
T<|>3[t,i];  in  these  transitions  SON^Ct*)  = NIL,  so  that  node  i has  no 
sons  at  time  t+.  Therefore,  it  is  left  to  check  only  that 
the  properties  are  preserved  for  fathers  of  node  i. 

If  i £ SONkCt-)  then  Or,  s . ) (t+)  > (n.  ,s . ) (t-)5(nk,skXt-) 
where  the  inequalities  follow  from  Lemma  A.l  b)  and  from  the 
induction  hypothesis  respectively;  so  the  properties  are 
preserved  for  all  fathers. 
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T12[t,i],  T22[t,l],  T22[t,i]  ; d^  si  and  possibly  are  changed;  no 

change  in  SONL.  Regarding  fathers,  the 
proof  evolves  as  for  T>3.  Regarding 
sons,  we  see  that 

Txy[t,i,(jil,n2],CS0Nl,S0Nl)]  (A. 47) 
where  Txy  = T12  or  T22  or  T22,  implies 
from  steps  B.1.1,  B.1.6,  B.5.1,  B.5.3, 
B.9.1,  B.9.2  in  Section  3.3.3,  that 

Ni(k)Ct-)=n2  , D.Qclt-1^- VkcSONl  CA.48) 
We  now  continue  this  part  of  the  proof 
for  one  node  JleSONl,  for  each  other 
son  the  proof  follows  in  the  same  way. 

From  b)  and  R2,  (A. 48)  implies  that 
3Tj  e [LPC(m,n21,t] 
such  that  si(TJil=S2[n2]  . Now,  if  on 
Cr^.t),  node  £.  stayed  at  S2[n2]  or 
performed  any  transition  except 
T21j>, Cn2,n2)J  , then  the  properties 
are  preserved.  Therefore,  it  suffices 
to  prove  that  node  i could  not  have 
performed  T21  on  (t^.t).  Suppose 
it  has  , i.e. 

T21[TlrA,(n2,n2)]  , tl^C^t)  CA.49) 
does  happen.  Then  by  step  B.4.1  in 
Section  3.3.3  we  have  n^Ci) (Tl^)=n2 . Now 


we  distinguish  between  two  cases: 
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If  D.CiKUJ  i - , then  by  b),  3 i2.e  [LPC(m,n2)  ,rl.) 


i v 1 v l 
such  that 


[t29 , i,SEND[n2,d,b,£)]  , d i « 


(A. SO) 


which  by  property  R2  implies  that  s.(t2.-)  = S2[n2] 
or  s^C^*)  = S2[n2].  But  CA.47)  says  that  i enters 
S2[n2]  at  time  t, which  contradicts  c)  iii). 

If  * 00 , then  for  some  time  ^2^  < tl^ 

[T24,i,SENDCn2,d=»,b,H)]  (A. SI) 

which  implies  that  s^Cf2i+)  = S3[n2].  But 

s^(t+)  = S2[n2]  and  t2^  < t,  which  is  impossible  by 

property  R3  and  Lemma  A.l  b). Therefore  (A. 49)  does  not 
happen . 

T32[t,i];  suppose 

T32[t,i,(nl,n2), (NIL.S0N2)]  (A. 52) 

happens . 

Regarding  fathers,  the  properties  are  preserved  since  by 
property  R3,  n2  > nl.  Regarding  sons,  then  by  b)  the 
above  implies  that  3t  e (LPC (m,n21,t]  such  that 

[t,*,  SEND(n2,d,b,il]  , i e S0N2  (A. S3) 

Now,  from  Lemma  A.l  bl,  n^Ct)  * n^C1)*  From 
property  R2,  n^C1)  ■ n2.  Now,  if  n^Ct)  > n2,  then 
Cn^.s^iCt)  > (n-.s.Kt*)  = Cn2,2). 

If  on  the  other  hand,  n^(t)  = n2,  then  the  same  argument 
as  for  T12,  T22,  T22  shows  that  node  SL  was  in  S2[n2] 
sometime  before  time  t and  could  not  return  to  Sl[n2] 
in  the  meantime,  so  that 

(n^.s^Kt)  * Cni,si)(t+)  (A. 54) 
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So,  the  properties  are  preserved  in  this  transition. 

In  addition  to  the  above,  since  here  there  is  a change 
in  SONL  from  NIL  to  i NIL,  we  have  to  show  that  a loop 
is  not  generated  by  this  change.  This  is  seen  from  the 
fact  that  every  node  k upstream  from  node  i at  time  t 
has 

Cnk» skl  (-t)s  C^i , s^)(t-}=  cm , 3) < (p2 , 2)  = (ik , s^(t+) 

where  the  first  inequality  follows  frpm  the  induction 
hypothesis. 

Also  every  node  q downstream  from  node  l has 
Cnq,sq)Ct]^ChJl,sJlKt}>(jii.si]  Ct+Mn2,2) . 

So,  the  reattachment  does  not  generate  a loop. 

T2 1 £ t , i] ; suppose 

T21[t,i,(n2,n2),(dl,dl),(SONl,SON2)l(pl,p2)]  CA.SS) 

happens . 

Regarding  fathers,  i.e.  if  i e SON^Ct),  then  from  d)  ii) 
it  follows  that  s^Ct)  = Sl[n2],  therefore 
Cni,silCt+l  = Cnk.sk)Ct). 

Regarding  sons,  i.e.  if  l e SON^ (£«■],  then  steps  B.4.1 
B.4.4,  B.4.11  in  Section  3.3.3  show  that 

N.UHt-)  - n2,  D.^Kt-l  ^ Vt  £ SONi(ti-) 

Then  from  b)  e [LPC(m,n2) ,t]  such  that 

[TA,i,SENDCm,d,b,i}]  , e SON.(t+)  (A. 56) 


with  m * n2  ■ 

Therefore,  from  Lemma  A.l  hi 
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This  completes  the  proof  for  the  properties  stated  in 
ii)  and  iii)  in  Theorem  3.1. 

We  new  turn  to  prove  Theorem  4.1:  To  prove  this  property  notice 
that  if  (A. 55}  happens,  then  p2  e SOfTCt+)  by  steps  B.4.11  in  Section 
3.3.3  and  B.4.14  in  Section  4.3.2,  and  if 
Cnp2,sp2Kt+]  = Cni,si)(.t+)  * (n2,i) 
then  by  steps  B.4.1,  B.4.4  in  Section  3.3.3  and  B.4.14  in  Section  4.3.2 
we  have 

Ni(p2)Ct-)  = n2  , D.(p2)(t-)  i - 

From  b)  ^ Tp2  £ [kPC(jn,n2) ,t]  such  that 

[ip2»p2,SEND(jn,d,b,i)]  (A. 57) 

with  m = n2  and  d ® dp2(.x  2)  = D^(p2)(t-)  - D'^p2. 

By  property  R2,  s^CTpj*)  = S2[n2],  so  by  c)  iii),  node  p2  could  not 
enter  again  S2[n2]  in  the  interval  of  time  (t  2+,t),  therefore 
dp2(t)  = CTp2+)  • But  by  steps  B.4.2,  B.4.6,  B.4.7  in  Section  3.3.3 
and  B.4.14  in  Section  4.3.2,  we  have 

diU+)  • di  * DiCp2)Ct-l  = dp2  (t)  ♦ D'.p2 

which  from  assumption  2 in  Section  3.3.2  implies  that 
dl  = di(t+)  > dp  O)  = dp2Ct) 

completing  the  proof  of  Theorem  4.1,  q.e.d. 

In  addition,  to  complete  the  proof  of  loop-freedom  property,  we  have 
to  show  that  the  possible  change  in  the  list  SOfT  of  sons  in  T21[t,i] 
does  not  generate  a loop.  We  prove  this  by  contradiction. 


Suppose  that  at  time  t+  a loop  is  closed  because  of  T21[t,i]. 

Since  by  the  induction  hypothesis  the  network  was  loop-free  until  time  t-. 
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I 


then  the  assumed  loop  must  contain  p^Ct*).  Denote  the  loop  as  the 

following  string  of  nodes:  i.  ,i9, . . . ,i  p=i,  i.  = p.Ct+1  i . is 

Li  x,  i i j q-t-i 

a son  of  i^  for  q = 1,2, . . . ,Z-1,  and  i^  is  a son  of  i at  time  t*. 

Observe  that  at  time  t- jti  . was  a son  of  i for  q = 1 ,2, . . . , (A-l)  too, 

but  ij  i SON\  Ct-)  from  the  induction  hypothesis.  In  addition,  from 

Theorem  3.1  ii)  and  3.1  iii)  we  see  that  around  the  assumed  loop  the 

concatenation  Ch,s)  is  nondecreasing,  so  it  must  be  constant,  namely 

(n.s)  = (n2,l)  at  t+  around  the  assumed  loop.  Clearly,  this  loop 

must  contain  a link  (ir,i  +1)  such  that  d^  S d.  at  time  t+,  and 

r r+  xr  1r+l 

Ci  , i .)  ft  Ci5.i1}  which  follows  from  Theorem  4.1.  We  have  already 

shown  that  [LPCCm,n2)  ,t]  such  that  s.  CTt)  = S2[n2],  so  by  c)  iii) 

11 

node  i^  could  not  enter  S2[n2]  again  between  t+  and  t.  Since  at  time 

t+,  s. Ct+)  3 Sl[n2],  then  T21[t.  i ] happens  at  some  time  t.  e CT,t) 

1 1j,  1 xi 

and  no  other  transitions  happened  during  the  interval  CT.t).  Using  the 
same  arguments, we  see  that  each  node  i^Gr=i.2, . . . ,£)  has  been  in  state 
S2[n2]  at  some  time  after  LPC(m,n2)  and  before  t,  has  not  entered  S2[n2] 
again  until  t,  -and  has  performed  T21[i^]  after  being  in  S2[n2]  and  before 
t,  performing  no  transitions  in  between.  While  entering  S2[n2] , each 
node  i (q='l»2, . . . ,£)  has  updated  its  d and  b , and  has  not  done  it 
again  until  t. 


Let  i.  , x.  be  the  last  time  before  t nodes  i , i . updated  their 
xr  1r+l  L 

d,  ,b.  ,d.  ,b.  Respectively.  (From  now  on,  slight  differences 

1r  xr  Xr+1  xr+l 

appear  in  this  part  of  the  proof  between  the  line  switching  and  the 
message  switching  models.  We  indicate  these  differences  when  applicable). 


We  now  claim  that  d.  Ct+)  S d.  (t+)  implies  r.  > t.  . This 
1r  r+l  r r+l 

is  because  that  if  d.  was  updated  after  x , then  it  means  that  at 

Vl  Xr 

v.  node  i , was  not  a son  of  i , and  became  one  on  (x.  ,t) , which  from 
1 r+l  r’  1 

r r 

Theorem  4.1  and  property  R9  imply  that  d,  Ct)  > d.  Ct)  contrandicting 

r xr+l 


■' -- . - . — _ , 
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our  assumption,  Therefore,  at  t.  , node  1 . is  a son  of  node  1 with 

r i r+i  r 

r 

d.  5 d.  . Clearly,  for  message  switching,  4>.  . >0  and  for  line- 

xr  xr+l  1r’1r+l 

switching  f.  >0,  the  latter  follows  from  the  fact  that 

1r,1r+l 

d.  5 d.  and  step  B.4.1.1  in  Section  3.3.3.  This  situation  is 
r r+1 

not  changed  at  least  until  time  t,  since  the  only  transitions  nodes 

ir  and  ir+^  can  perform  in  (r^  ,t)  is  T21,  and  since  at  time  t node 

r 

i . is  still  a son  of  node  i . 
r+l  r 


Let  tl  < t be  the  time  node  if  performs  T21 [tl ,ir, Cn2,n2) ] , then 
at  tl-  we  have  for  message  switching 


nai  a /^i  < i 

1r’ir+l  r r’V+l 

for  line-switching 

na.  . < f ' . ( f. 

i ,i  . l ,i  , l ,i  . 

r r+1  r’  r+1  r’  r+-l 

otherwise,  at  time  tl+,  i i SON^  contradicting  our  assumption. 

r 

From  step  B.4.8.2  in  Section  3.3.3  we  also  have  at  tl-: 

nai  ,i  , * ^ i (W  - di  1 
r ’ r+1  r r 

So,  at  tl- 

for  message  switching 


(A. 58) 


(A- 59) 


(A. 60) 


,[,irar.l>  - - *1.4. 

r r r r+1 

for  line  switching 

"tDi  - di ) < fi  ,i  , 

r r r r+1 

However,  on  the  interval  of  time  (t.  ,tl)  all  the  above  quantities  are 

1r 

not  changed.  Therefore  (A. 61),  (A- 62)  hold  at  also,  which  implies 

r 

that  + 1 at  Csee  steps  B.1.5,  B.5.3,  B.3.2  in  Section  3.5.3). 


(A. 61) 


(A. 62) 


We 


now  notice  that  node  i , entered  state  Sl[n2]  only  after 

r- 1 
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receiving  a message  MSG(n2,d/“,b}  from  node  ir,  so  node  ir  could  not 

become  a son  of  i 1 in  T21[if_1]  since  if  was  blocked.  Therefore  node 

i j set  «-  1 while  entering  S2[n2]  (because  either  iy  was  a son 

or  became  one  in  T321.  Following  the  same  argument  we  move  upstream  on 

the  assumed  loop  from  i to  i.,  and  see  that  b.  = 1 at  time  t.  But 

r x 

this  says  that  was  not  a son  of  node  i and  became  one  at  t+  although 
it  was  blocked  at  t.  Step  B.4.6  in  Section  3.3.3  does  not  allow  this 
to  happen  asserting  a contradiction.  q.e.d. 

f)  i)  During  CTl.t),  no  link  is  brough  up  by  node  i because  of  Lemma  A. 4. 

Now,  suppose  there  were  failures,  let  t3  be  the  first  time  on 
<jl,t)  such  that 

[t3,i,FAILCk)];  (A. 63) 

Then  node  i performs  either  T23[r3,i, Cnl.nl)]  or  T22[t3 ,i, (nl ,nl) J 
with  nl  ■ m.  In  either  case,  d)  i)  shows  that  to  exit  S3[nl]  or 
S2[nl]  and  enter  to  S2,  one  has  to  increase  n^,  so  that  it  is  impossible 
that 

T21[t,i,  Cnl.nl)]  (A. 64) 

happens.  So  no  failures  can  occur  on  Crl.t),  q.e.d. 

ii)  Consider  the  following  sequences  of  nodes  and  instants: 
i ■ ig,  1i» • • • »iq  = SI^^ 
t = tQ  > ^ > t2  > > tq 

such  that 

Wltu.iu*  GM2),  (*,S0N2.  )]  (A. 65) 

happens,  where  n2  - m,  iu+1  e SON2 . (t  +),  u = 0,l,.,.,q-l. 

u 

Such  sequences  must  have  existed  if  Tty2[t,i,  0lsn2)]  happened. 


Also  by  e)  the  seqeunce  of  nodes  contains  no  loop  until  PC(m). 
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Now,  assume  that  * Tu  e [tu'PC  0,111  SUcb  that 


T21[Tu,iu,Cn2,n2l] 


(A. 66) 


(A. 67) 


happens  for  u*0.  We  want  to  show  that  this  assumption  leads  to 

the  fact  that  * Tu+1  e IVi.PCW]  suc^  that 

T21[t  . ,i  . Ch2,n2)] 

1 u+1  u+1,  1 

happens  (remember  u»0) . 

Suppose  there  existed  such  , then  it  follows  from  f)  i)  that 

Fx  ^u^Vl1  “ UP  (since  Fi  /V^u+l1  = UP1- 
u+1  u+1 

The  next  step  of  this  proof  is  to  show  that  ^u10tu+1_1  ^ m = n2- 

u+1 

To  do  this  we  must  show  that  ^t2u+i  < Tu+1  suc^  that 

Cx2u+l'iu‘SENDCp2*d>b’iu+l)1’d  = " (-A.68) 

and  that  ^ t3u+1  g [PC(jn)  »TU^.1D  such  that 

tT3u+l'iu*SENDCjl2>djb'iu+l11  ’ d * “ ’ (A.  69) 

For  t2u+j  < tu>  it  follows  that  T]  such  t2u+1  from  properties  R2 

and  R3  (since  s^^  (t2u+1+)  » S3[n2]  and  it  can't  be  that 

si  (t  ) * S2[n2]).  For  t3  . < tu,  it  follows  from  property  R2 
1u  u u+ 

and  c)  iii)  (.since  s.  (t3  .♦)  a S2[n2]  and  it  can't  enter  S2[n2] 

xu  u+ 

again  at  t again) . 

For  t2  , ■ t or  t3  ,■  t , it  follows  from  the  fact  that 
u+1  u u+1  u 

i , e S0N2.Ctu)  and  iy  does  not  send  a message  to  its  sons  in  Ti|>2. 

1u 

For  t2u+1  e (tu,PC(m))  or  t3u+1  e (tu,PC(m)),  the  only  possibilities 
for  iu  if  T21[iu]  does  not  happen  are:  to  stay  in  S2[m]  or 
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T21tVrVi'Cn2’n2i] 

is  impossible  (by  step  B.4.1  in  Section  3.3.3) 

Regarding  the  proof  by  induction  (increasing  u by  1)  shows  that 
it.  such  that 


(A. 70) 


T21[;qISINK,(n2,n2)]  , n2  « m 


(A. 71) 

happens,  which  contradicts  the  assumption  that  there  is  a proper 
completion  at  time  PC(m). 

This  proves  the  first  part  of  f)  ii).  The  second  part 
of  f)  ii)  follows  because  T21[Tl,i, (n2,n2)] , n2  =in  is  not  possible 
if  T32[x2,i, Cn2,n2)]  or  T22[x2,i, (n2,n2)]  happen,  q.e.d. 


g)  If 


[x,k,SEND(m,d,b,i)]  , d / » 
then  F^Ci)Cx)  ■ UP  and  by  property  R2  either 
T'^2[x,k,  (^,n2) ] , n2  = in 
or 

T21[xl,k,  (n2,n2)]  , n2  = m. 


(A. 72) 


(A. 73) 


(A. 74) 


If  Tt>2  then  f)  ii)  implies  that  3r2  e (x,PC(m))  Such  that 

T21[x2,k,(p2,n2)]  , n2  - m (A. 73) 

and  Fjc(i)Cx2)  * UP.  Therefore  T21  happens  at  node  k a<  some  time 
(xl  or  t2).  Call  this  time  n.  We  have  then  N^(.i)(n)  a m.  By  b) 
either  3x3  £ [PC(m),n]  such  that 

[x3,  i,SEND(jn,d,b,k) ] , d * • (A. 76) 

or  x4  < n such  that 

[ x 4 , i , SEND  (Jn , d , b , k)  ] , d - - 


(A. 7") 
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but  by  property  R2,  this  means  that  node  i is  at  t4  < PC(m)  in 
S3[m]  or  is  at  PC(ra)  < t3  < PC(jn)  in  S2 [in] . 

If  the  first  holds,  node  i will  stay  in  S3[m]  at  least  until 

PC(m).  If  the  latter  holds,  then  by  f)  ii)  it  must  perform 

T21[i,  (n2,n2)] , n2  » m before  PC(jnl-  But  since  at  time  t it  still 

has  fTOOCt)  * m,  (JO  (t)  t • or  has  not  received  yet  the  message 

by  time  t,  c)  i)  implies  that  node  i could  not  perform  T21[i,  (n2,n2)] , 

n2  ■ m before  time  t.  Therefore  it  will  perform  it  later,  q.e.d. 

Proof  that  al  holds  at  time  PC  (in).: 

i)  Node  i cannot  be  in  S2[m]  because  of  f)  ii)  and  c)  iii) . It 
can't  be  in  S2[m]  because  it  must  have  been  in  S2[m]  before 
and  because  of  f)  ii). 

ii)  Take  t - PC(m)  in  g) . It  follows  then  that  s^PCCm))  * S2[m] 
or  S3  [in]  but  f)  ii)  and  c)  iii)  imply  that  s^(PC(ra))  t S2[m]. 

iii)  Follows  by  contradiction,  since  if  at  PC(m) 

^00  CPC  (in) ) = m , DiCklCPC(p))  i - 
it  follows  by  taking  t ■ PC(in}  in  g)  that  s^(PC(m)l  * S2[m] 
or  S3  [in] , q.e.d. 

This  completes  the  proof  of  Theorem  A.l. 


k 
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APPENDIX  B 

In  appendix  A we  have  proved  Theorems  3.1,  3.2,  3.3  and  4.1. 
This  appendix  is  devoted  to  proofs  of  the  remaining  statements  of  our 
work,  namely  Theorems  3.4,  3.5  and  4.2.  The  proofs  are  organized  as 
follows:  Lemma  B.O  is  preliminary  and  shows  that  on  any  link  (i,£)  the 
only  two  "stable  situations  are  (F^(S.)  = F^Ci)  = DOWN}  or 
{F.(£)  t DOWN,  F.(i)  t DOWN}.  Lemmas  B.l  and  B.2  prove  Theorems  3.5, 
Lemma  B.3  proves  Theorems  3.4  and  Theorem  4.2  is  proved  by  the  series  of 
Lemmas  - B.4  to  B.7. 

Lemma  B.O 

If  F^(i)ftl)  * DOWN,  F1(i)(tl)  j*  DOWN,  then  in  finite  time  after  tl  we 
have  either  { F± C«-)  = Fa(i)  = DOWN}  or  {F± (A)  / DOWN,  F^Ci)  t DOWN}. 

Proof 

F£Ci) Ctl)  jt  DOWN  means  either  F4(i)(tl)  = READY  or  F^CiKtl)  = UP. 

If  Fa(i)(tl)  = READY,  then  nodes  i and  l arrived  at  this  situation  from 

{FjL(S.)  = FA(i)  = DOWN}  or  (F.(Jl)  = F^fi)  = READY}  or 

(F.(A)  = UP,  F.(i)  = READY}.  Then  assumptions  9 in  Section  3.3.2  imply 
the  assertion. 

If  F^Ci) (tl)  = UP,  then  nodes  i and  i arrived  at  this  situation  from 

(Fi(Jl)  = DOWN,  F^(i)  = READY}  or  (F.(H)  = F^i)  = UP}  or 

(F.(£)  = READY,  F.(i)  = UP}.  In  the  first  case,  the  discussion  reduces 

X x» 

to  the  first  part  of  the  proof,  whereas  for  the  second  and  the  third 
cases,  assumption  9. a)  in  Section  3.3.2  proves  the  assertion. 


k.  . 
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Lemma  B 1 

Theorem  3 5 a) . 

Proof 

Clearly,  n^(tl-)  < m2  for  all  i (property  R8).  Therefore  (3. 16)  may 
happen  only  at  or  after  tl.  Let  us  now  define  four  sets  of  nodes: 

A(t)  ■ { i 1 l e L(t)  and  i effected  (3.16)  with  t2^  < t>, 

B(t)  » { x | i t L (t)  and  i i A(t)  }, 

A'(t)  « { i | i e A(t)  and  i has  a potentially  working  link  to  a node  in  B(t)}, 

B'-(t)  = { x | i e B(t)  and  i has  a potentially  working  link  to  a node  in  A(t)}. 

If  there  is  an  instant  t2  such  that  A(t2)  = L(t2),  then  the  proof  is 
complete.  Otherwise,  for  a given  instant  t3,  we  will  show  (by  contradiction) 
that  there  is  an  instant  t,  t3  < t < ® such  that 

A(t)  O A(t3)  and  A(t)  f A(t3).  (B.  1) 

Hence  by  induction,  the  set  A(t)  keeps  growing  until  it  equals  L(t). 

Since  there  are  no  pertinent  topological  changes  and  since  all  nodes 
i e ACt)  have  ni(t)  = m2,  property  Rll  implies  that  the  set  A(t)  is  non- 
decreasing as  t increases.  Therefore,  to  proved  part  i)  of  Theorem  3.5  a) 
it  is  sufficient  to  show  that  the  following  cannot  hold: 

•yt  > t3  , A(t)  = A(t3)  i L(t)  (B.  2) 

We  contradict  (3.2)  by  the  following  three  claims: 


1 
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CLAIM  1 

If  (B.  2)  holds,  then  3 E Ct3,®)  such  that  vJ  e B'Ct4), 

3 t4  <■  t4  such  that 

[t4jk,j,MSG0»2,dj«-,k],  (B.3) 

k such  that  k e A' (t4)  and  F^(j)(t3)  = UP  (i.e.  all  nodes  of  B' (t4) 
receive  m2  in  finite  time  from  all  their  neighbors  in  A'(t4)). 

Proof  of  CLAIM  1 

At  time  t2^  *"  t3,  node  i c A'  (J2^1  performs  (3.16).  For  links  (i,£), 
where  i e A'(t2^),  Z t B ' Ct 2 i ) and  Fi (L) (t2^+)=  UP,  observe  from  steps 
B.1.8,  B.5.3,  B.8.9,  B.9.2  in  Section  3.3.3  that  if  Z i S0NV(t2^),  then 

[t2.  ,i,SEND(m2,d*®,Jl)]  . (B.4) 

Notice  further  that  for  each  node  k e S0NL(t2^)  we  know  from  Theorem 
A. 1 e)  that  k ? B^^).  Observe  also  that  since  no  pertinent  topological 
changes  occur,  property  Rll  insures  that  for  all  nodes  Z,  F^(&)  cannot  be 
changed  from  or  to  DOWN  after  t2^  for  i e A'(t2^).  It  means  that  if 
FiU)Ct2i-)  * DOWN  then  F.(£)(t)  = DOWN  1/t  * t2.  and  if  Ft (A) (t2.-)*READY 
or  UP,  then  F.(i)(t)  * UP  ft  >.  t2i  (see  steps  B.1.8,  B.S.3,  B.8.8,  B.9.2 
in  Section  3.3.3).  Therefore,  from  assumption  3 in  Section  3.3.2  it  is 
insured  that  the  message  in  (B.4)  arrives  at  l in  finite  time.  So,  there 
is  a time  t4  for  which  all  nodes  j that  were  in  B'(t2k)  for  some  k, 
either  are  not  in  B'(t4)  anymore  or  have  received  MSG(m2 ,d^®,k)  at  time 
t4jk  < t4  from  all  nodes  k such  that  k e A'(t4)  and  F^(j)(t3)  = UP. 


1 


q.e.d.  CLAIM  1. 


- ] 24  - 

Notice  now  that  B'  (t4)  cannot  be  emplty,  since  then  C.B . 2 J is 
contradicted. 

Let  t5.  = max(_t4  , ) for  a node  j e B'(t4)  where  t4..  is  as  defined 
J ^ J * J * 

ir  CLAIM  1 There  exists  such  time  t5^  < « because  of  Claim  1 and  since 
there  is  a finite  number  of  nodes  in  the  network,  Cin  words,  t5^  is  the 
time  that  a node  j e B'  (_t4)  has  received  MSGCm2,d^®}  from  all  its 
neighbors  in  A'(.t4)]. 

Now,  if  gj  t B’  (t4)  such  that  yk  £ SON^  (t5, ) , k e A'(.t4),  then  from 
steps  B.l.l,  B.1.2,  B.S.l,  B.5.2,  B.9.1  jn  Section  3.3.3. 

T*2[t5jfj.(*.m2)]  (B . 5) 

happens,  contradicting  CB.2)  q.e.d. 

Therefore,  we  now  assume  that  Vj  e B'tt4)  the  following  cannot 
hold:  {yk  e S0NjCt5j)  , k e A'Ct4)>. 

CLAIM  2 

If  j £ B'Ct4)  and(yk  e S0N^Ct5^)  , k e A'(t4)}does  not  hold,  then 
yt  > t4j  the  following  cannot  hold :{  V k t SONj  (t) . k e A'(t4)l. 


Proof  of  CLAIM  2 


Suppose  there  is  time  t > tS^ 
Then,  for  the  first  time  after  t5.. 


such  that{  y k e SON^Ct),  k e A'(t4)}. 
it  holds 


Txy[t,j,CS0Nl,S0N2]J  , t > tS.. 


CB.b) 
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or  Cx[t , j , (S0N1 ,S0N2) ] , t > t5^  (B.7) 

happens  with  SON1  ^ SON2. 

If  T22  or  T12,  then  SON1  = SON2,  so  these  transitions  do  not  hold  here. 
If  Ti^3,  then  S0N2  = NIL  / k,  hence  cannot  happen. 

If  T21,  theni^q,  (q)  = n^  < m2,  hut  (k)  (tl  ■ m2,  hence  T21  cannot 
happen. 

If  T32 , then  S0N1  * NIL  and  T32{t, j , (j^,m2)]  happens,  contradicting 
(B.2),  hence  cannot  happen. 

If  T22  or  Cl  or  C2,  then  exactly  one  node  is  delected  from  S0N1, 

call  it  i.  After  this  node  is  deleted,  we  assumed  that  ^k  e S0N2,  then 

k e A'(t3),  therefore  node  j will  effect  T22[t,j  , (i(/,m2)]  or 

Tl2[t,j,  O', m2)]  since  n^  < m2,  which  contradicts  (£.2),  hence 

cannot  happen. 

q.e.d.  CLAIM  2. 

CLAIM  3 

In  finite  time,  all  nodes  i e B(t4)  will  effect  T^3[i, (^.m)]  with 
m < m2, without  effecting  T32  thereafter. 


Proof  of  CLAIM  3 

We  know  from  Section  3.3.3  that  n^  can  be  updated  only  in  transitions 
T'^2  and  T>3.  For  all  nodes  i e B(jt4),  T4'2[i,  O', m2]]  does  not  occur, 
otherwise  (B.2)  is  contradicted.  Also  Tt|i3[i,  0)>,m2)]  does  not  occur  for 
all  nodes  i e B(t4)  because  no  message  of  the  type  MSG(m2,d«“)  is  generated  in 
the  network  since  there  are  no  pertinent  topological  changes.  Therefore, 

= j 


f 

I 

\J.i  z B(t4)  and  'j  t > t4,  then  n^(t)  < m2.  (B.8) 

After  time  t4,  no  update  iteration  with  m < m2  is  started  by  the  SINK 

(since  the  SINK  has  started  an  iteration  with  m = m2  before  t4) . By 

Theorem  3.2  ii)  it  implies  that  the  number  of  messag«with  d i 00 

generated  by  the  nodes  that  belong  to  B(t4)  is  finite  (remember  we 

deal  with  a finite  number  of  nodes  in  the  network).  Similarly,  since 

the  number  of  links  is  finite,  the  number  of  FAIL  messages  is  also 

finite.  Let  t'  be  the  time  after  all  these  messages  are  generated  and 
mx 

received.  Define  t = max(t4,t'  ).  Clearly,  -j-i  z B(t4),  T32 [ i] 
cannot  occur  after  t (since  all  FAIL  messages  and  MSG(d#°°)  have 
been  already  received) . 


We  now  define  the  following  set  of  nodes: 

B(t)  = { i | i e B(t4)  and  SON.(t)  = NIL). 

If  B(tmx)  = B(t4)  then  q.e.d.  claim  3.  Otherwise,  there  are  nodes 

k e B(t4)  and  k i BCt^).  A11  these  nodes,  after  a sufficiently  long 

period  of  time  - t*  , will  not  have  sons  which  belong  to  B(t  ) 
mx  mx 

(since  nodes  in  ^(t^)  effect  Txy[i .SEND (m,d=°°) ] when  SON^  is  set  to 
NIL,  therefore  they  are  deleted  from  the  list  of  sons  of  every  k z B(t4) 
and  k t BCt^)).  Since  there  are  no  loops  in  the  network,  at  t*mx 
there  is  a node  i e B(t4)  and  i f ^as  no  son  also  in 

the  set  of  nodes  (k|  k e B(t4)  and  k t BCtmx)}.  By  CLAIM  2, 
this  node  neither  has  all  its  sons  in  A'(t4).  Consequently,  at  t*  , 
this  node  has  no  sons  at  all,  so  s^(t*mx)  = S3  and  it  cannot  leave 
S3  thereafter  (since  all  messages  MSG(d^“)  has  already  been  received).  By 
induction,  the  set  of  nodes  B(t)  grows  until  it  equals  B(t4). 

q.e.d.  CLAIM  3. 
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The  proof  of  Theorem  3.5  a)  i)  is  completed  as  follows: 


Consider  a node  j e B'Ct4).  Define  t3^  to  be  the  time  at  which 
Tii<3[t3j,j]  occurs  by  CLAIM  3.  But, 

if  t3j  < t5j , then  T32[t5j,j]  happens, 
if  t3j  > tSj',  then  T32[t3..,j]  happens, 

and  t3j  ft  5t^.  since  j processes  the  messages  one  at  the  time.  This 
contradicts  (B.2).  So  by  induction,  the  set  ACt)  keeps  growing  until 


it  equals  L(t) . 


q.  e . d. 


To  prove  part  (ii)  of  Theorem  3.5  a},  we  investigate  further 
the  situation  in  L(t2)  at  time  t2.  Observe  that  since  all  nodes 
i £ LCt2)  have  ni  = m2,  and  since  no  pertinent  topological  changes  occur, 
it  follows  from  Rll  and  Lemma  B.O  that  for  any  link  (i.^)  such  that 
i £ L(t2)  and  1 t LC.t2),  it  cannot  happen  that  at  time  t2  we  have 
F.(Z)  3 DOWN,  F.(.i)  ft  DOWN.  Also  F.(A)  = READY  is  impossible,  because 
lack  of  pertinent  topological  changes  imply  that  F^ (JO  = READY 

as  well,  and  then  by  steps  B.1.8,  B.5.4,  B.8.8,  B.9.2  in  Section 
3.3.3  F^ (£)  (t2^+)  =UP,  therefore  F^(fc)Ct2]  = UP.  Consequently, 
for  links  connecting  nodes  in  L(t2),  the  only  possibilities  at_ 

time  t2  are  {F,  (£)  3 F„(i)  3 DOWN}  or  {F.  C/0  = F„(i)  3 UP),  hence  part 

1 A>  1 A 

ii)  of  Theorem  3.5  a}  is  proved. 


Next  assuming  Theorem  3.5  a)which  was  proved  by  Lemma  B.l,  we  now 


prove  Theorem  3.5  b). 
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Lemma  B.2 

Theorem  3.5  b). 

Proof 

We  first  prove  part  i)  of  Theorem  3.5  b)  by  showing  that  there  is 
PC(ml)  after  tl  and  that  there  is  no  PC(ml]  between  tl  and  t2. 

Lack  of  pertinent  topological  changes  insures  that  after  entering 
S2[ml]  at  t2;,  each  node  i e L(t2)  can  only  perform  transitions  between 
states  Sl[ml]  and  S2[m2].  Furthermore,  by  Theorem  3.1  x)  notice  that 
after  t2,  these  nodes  form  a loop-free  pattern  (lattice)  with  the  SINK 
the  only  terminating  node.  Consider  a time  t',  t'  > t2.  L(t'l  = L(t2) 
since  there  are  no  topological  changes.  Also,  by  Theorem  3.2  iii),  if 
a node  i e L(t2)  enters  S2[ml]  after  t2,  then  PC(ml)  has  occured  after  tl. 

1.  If  s^Ct')  = Sl[ml],  i e L(t'),  then  there  exists  t3,  tl  < t3  < t' 
such  that  T21 [t3 ,SINK, (ml ,ml) ] happened  (^ince  SINK  z L(t')); 

2.  Otherwise,  consider  a node  k e L(t'l  such  that  s^Ct')  = S2[ml]  and 
^ j , if  k e SON^(t'),  then  s^Ct')  = Sl[ml]  (notice  there  can  exist 
no  such  a node  j).  Such  a node  k must  exist  if  not  all  the  nodes 
are  in  SI [ml].  Classify  the  neighbors  of  k into  the  two  following 
sets  of  nodes: 

A ■ (i|  F.(k)Ct’l  ■ UP  and  s . (t ' ] = SI  [ml]}, 

B =*  U|  F.Ck)Ct')  = UP  and  s.Ct'}  = S2[ml]>. 


At  some  time  in  the  interval  [tl.t'j,  each  node  i e A has  sent  message* 
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MSG (ml, d^®)  to  all  its  neighbors,  namely  to  all  nodes  q such 
that  F^CqJCt'l  = UP.  Also,  at  some  time  during  the  same 
interval,  each  node  i e B has  sent  such  messages  to  all  their 
neighbors  except  sons,  namely  to  all  nodes  q such  that  F^(q)(t')  = UP 
and  q t SON^Ct'}.  However,  k is  not  a son  of  any  of  those  nodes 
in  B (since  it  is  a son  only  of  nodes  in  A).  Hence, by  3 in  Section  3.3.2 
node  k will  receive  messages  MSG (ml ,d?“0  from  all  its  neighbors, 
at  a finite  time,  say  t4.  Then: 


2.1:  If  s,(.t4+)  = S2[ml],  then  ^Ji  with  F^U-KM)  = UP  such  that 
Ci i Ct 4 ) = NIL,  which  implies  that  T21 [k , (ml ,ml) ] happened 
in  the  interval  [tl,t4],  hence  by  Theorem  3.2  lii) , PC(ml) 
occured  between  tl  and  t4. 

2.2:  If  s,  (t4+)  = Sl[ml],  then  by  induction  PC(ml)  will  happen  in 

. 

finite  time  after  tl. 


We  show  next  that  PC(ml)  cannot  happen  in  [tl,t2].  Suppose  t5 

is  the  first  time  PC (ml)  occurs  after  tl  and  t5  < t2.  It  means  there 

is  a node  k e L(t2)  such  that  t2^  > tS.  Also  there  exists  a node 

j e L(.t2)  such  that  F^(.k)(t2^)  = UP  and  since  there  are  no  pertinent 

topological  changes  Fj(k)(t5)  = UP  too.  So,  node  j has  sent  to  k a 

message  MSG(ml,d^)  in  the  interval  [t2.,t5].  If  at  time  t5  the  message 

] 

is  on  its  way  to  k,  then  by  Theorem  3.3  ii),  s.^  = S3  [ml]  which  contradicts 
the  lack  of  pertinent  topological  changes.  If  the  message  has  arrived 
to  k before  tS,  then  at  time  t5  either  n^  < ml  or  s^  = SI [ml]  (since 
k has  not  entered  S2[m2]  yet),  and  by  Theorem  3.3  iii)  for  all  nodes  i 
such  that  F^(i)  = UP,  including  j,  it  cannot  happen  that 
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(Nk (i)  = ml,  Dk(i)  * . Therefore, 

means  that  t5  > t2. 


such  a node  k does  not  exist,  which 


q.e.d.  i) 


Furthermore,  we  notice  that  since  there  are  no  pertinent  topological 
changes,  we  have  LCt2)  = LCt3),  and  according  to  Theorem  3.1  i)  the 
Routing  Graph  of  these  nodes  forms  a loop-free  pattern  with  the  SINK 
the  only  terminating  node. 

q.e.d.  iii) 


Finally,  looking  at  the  situation  in  the  network  at  time  t2  as 
described  in  Lemma  B.l,  and  for  all  t e [t2,t3],  we  observe  that  for  all 
links  (j. , H)  for  which  FiWCt2)  = UP  we  must  have  F^Ht)  ■ UP  and  if 
F . (JO  It 2)  - DOWN,  then  we  must  have  F^Kt)  = DOWN.  This  completes 

the  proof  of  ii) . 

q.e.d.  Theorem  3.5  b) . 


Lemma  B . 3 

Theorem  3.4 

Proof 

From  Section  4.3.2  we  know  that  a new  iteration  T21 [tl .SINK, (ml ,ml) ] 
can  start  only  if  all  previous  iterations  with  the  same  counter  number 
ml  were  properly  completed.  Since  iteration  counter  numbers  are 
nondecreasing,  the  first  iteration  with  ml  has  been  started  at  some  time, 

say  t ' , by 
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T12[t',SINK,0n0,ml)]  , ml  > mO.  (B.9) 

This  transition  satisfies  the  conditions  of  Theorem  3.5.  Hence  in  a 
finite  time,  sa / tO,  the  iteration  is  properly  completed,  and  from  Section 
3.3.3  tO  > tl,  q.e.d.  Theorem  3.4  a).  Also  L(tO)  forms  a loop-free 
pattern  (.lattice}  with  the  SINK  the  only  terminating  node,  and 
n^  = ml,  i e L(tO),  and  since  there  are  no  pertinent  topological  changes, 
for  all  t >,  tO  we  have: 

1.  H(t}  = L(t)  ■ L(tO)  q.e.d.  Theorem  3.4  b). 

2.  By  Theorem  3.1  i)  all  nodes  i e L(t)  form  a lattice  with  the  SINK 
the  only  terminating  node,  q.e.d.  Theorem  3.4.d  ii). 

We  prove  Theorem  3.4  c)  by  induction:  First,  we  notice  that  since 
there  are  no  pertinent  topological  changes,  then  all  nodes  i t I.(tl) 
can  perform  only  transitions  between  Sl[ml]  and  S2[ml]. 

We  now  define  two  sets  of  nodes: 

A(t)  * { i 1 i e L(t)  and  i effected  T12[t2i , i , (ml ,ml)] , with 
tl  S t2 . S t } ; 

l * 

B (t } 3 { i | i e L(tl  and  i i A(t}}. 

tThe  induction  is  done  over  the  set  A(t}  and  we  want  to  show  that  it  grows 

until  it  equals  L(t).  Clearly  A(tl)  = SINK.  Assume  the  set  A(t  ) 

contains  several  nodes  for  t >,  tl.  Take  a node  k e B(t  ) which  all  its 

r r 

sons  belong  to  ACtr)  at  t (there  must  exist  such  a node  since  the 
network  is  loop-free).  Node  k can  change  its  list  of  sons  only  via  T21, 
so  it  does  not  change  this  list  at  least  until  k enters  S2 [ml j . At  t + 
all  nodes  in  A(tf)  have  sent  messages  MSG(ml  ,d?“)  to  all  their 


Ll 
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neighbors,  except  possibly  their  sons.  However,  node  k cannot  be  a son 
of  any  of  the  nodes  in  A(tr)  Csince  the  network  is  loop-free).  By  3 in 
Section  3.3.3  node  k receives  messages  MSG(ml,d^®l  from  all  its  sons  in 

finite  time,  say  t2^.  Therefore  at  t2^  node  k performs  (see  step  B.1.1 
in  Section  3.3.3). 

T12[t2k,k,(ml,ml)] . (B.10) 

We  now  can  add  node  k to  A(t2kl  and  delete  it  from  B(t2k).  By  induction, 
the  set  A(t)  keeps  growing  until  it  equals  L(t),  q.e.d.  Theorem  3.4  c) . 

Theorem  3.4.  d)i)  follows  directly  from  Lemma  B.2  by  assuming  Theorem 
3-4.  c) . q.e.d.  Theorem  3.4. 

Theorem  4.2  will  be  proved  by  Lemmas  B.4,  B.5,  B.6  and  B.7.  Lemma  B.4 
is  preliminary  and  is  used  to  simplify  the  following  proofs.  Lemma  B.5  deals 
with  the  case  when  a node  in  S2  or  S2  sends  a REQ(ml)  message.  Lemma  B.6 
proves  the  Theorem  for  the  case  where  there  is  a node  in  state  S3 [ml]. 

Lemma  B.7  is  similar  to  Lemma  B.5  but  for  SI. 

Lemma  B.4 

If  a REQ(ml)  is  generated  in  the  network,  then  either 
all  nodes  j in  the  entwork  have  n^  $ ml  and  REQQnl)  is  processed 
only  by  nodes  having  n^  = ml 
or 

a REQQnl)  arrived  at  the  SINK. 


r 
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Proof 


By  Theorem  3.1  ii)  and  from  Section  4.3.2,  REQ(ml)  is  not  received 
by  a node  i with  ni  < ml . On  the  other  hand,  if  there  exists  a node 
i with  n^  > ml,  the  SINK  started  an  iteration  with  m > ml ; this  is 


equivalent  to  the  arrival  of  REQ(ml)  to  the  SINK, 


Lemma  B.5 


If  a node  i sends  REQ(ml)  while  sfS2[ml]  or  s.^  = S2[ml],  then  a 
REQ(ml)  arrived  or  will  arrive  at  the  SINK  in  finite  time. 


q.e.d. 


Proof 


Consider  the  following  sets  of  nodes  and  intervals  of  time: 


i = iQ  e Aq,  Aj,  A2,  ...,  As  = SINK 


TIM  > TIM.  > TIM-  > . . . > TIM 
o 1 2 s 


such  that 


T*2[t.  ,ir,(*,n2),O,S0N2.  ),(*, p2.  )] 


B.ll) 


happens,  where  n2  = ml,  t^.  e TIMr>  ir  e and  S0N2i  C^=U+1Aa,p2 . e S0N2.  , 

r r r r 

r=0,l,...,s.  Such  sets  of  nodes  and  intervals  of  time  must  exist  if 

s 

s^  = S2[mll  or  s.  = S2[mll.  There  is  no  loop  in  the  set  of  nodes  U A 

a=0  a 

at  all  times,  otherwise  Lemma  B.4,  Theorems  3. 1,3. 2 or  3.3  will  be 
contradicted. 


The  proof  proceeds  by  induction.  We  know  that  i sends  REQ(ml) 
to  p2^  (unless  it  has  lost  it)  while  being  in  S2[ml]  or  S2[ml]. 
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Set  r=0.  Suppose  that  at  time  t2^  t.  , node  i sends  REQQnl)  to 

r r 

its  preferred  son  i = p2.  e A with,  q » r +1.  The  case  when  p2.  = NIL 

q xr  q 1t 

is  discussed  later.  Suppose  also  that  during  the  interval  of  time 

[t.  ,t2i  ] node  i performs  no  transition  except  possibly  T22  or  C2. 
r r 

Then  after  t.  , the  first  transition  executed  by  node  i could  be: 

1 q 

q H 

T22[i^] : q.e.d.  by  Lemma  B.4  (since  in  T22  node  i^  strictly  increases 
its  node  counter  number  from  ml). 

T22[iqJFAIL(£-j] : then  node  i detects  a failure  and  sends  REQQnl) 
to  its  preferred  son  while  being  in  S2[ml]. 

T21[i  ]:  this  transition  is  executed  only  after  receiving  a message 
from  iy.  Such  a message  is  sent  by  iy  when  T21[ir]  occurs, 
i.e.  after  iy  has  sent  the  REQQnl).  FIFO  at  node  i^  shows 
that  i will  receive  and  therefore  send  REQQnl)  to  its 
perferTed  son  before  T21[i^]  occurs,  i.e.  while 
si  = S2[ml] . 

q ~ 

T23[i^]  or  T22[i^,MSGCdE“)] : in  this  case  there  exists  a node  i^  £ 
l > q such  that  T22[i^,FAIL]  occurs  and  i^  sends  REQ(ml) 
to  its  preferred  son  while  being  in  S2[ml] . 

If  iq  performs  no  transition,  then  it  sends  REQOnl)  to  its  preferred 
son  while  s^  * S2[ml]. 

xq 

Thus,  by  induction, increase  r by  q , a string  of  nodes  is  established, 
in  which  each  node  sends  REQQnl)  to  its  preferred  son,  and  if  for  each 
node  i in  the  string,  p^  + NIL  then  REQQnl)  arrived  or  will  arrive  at  the 
SINK.  Finally  we  check  the  possible  case  that  one  (or  more)  of  the  nodes 
of  the  string  described  above  has  p^  * NIL.  In  such  a case,  since  while 
entering  S2 [ml ] each  node  in  this  string  has  p.  = NIL  (as  determined  by 
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step  B.1.10,  B.5.3,  B.8.12,  B.9.2  in  Sections  3.3.3  and  4.3.2)  then  i 
lost  its  p^  after  all  nodes  downstream  from  it  have  entered  S2[ml]. 
Therefore,  there  exists  a downstream  node  q which  detects  a failure  and 
T22[q,FAIL]  happens  at  that  node  Csince  the  network  is  loop-free)  and 
it  sends  REQGnl)  while  being  in  S2[mlJ  or  S2[ml].  Induction  asserts 
that  REQGnl)  arrived  or  will  arrive  at  the  SINK. 

q.e.d. 


Lemma  B.6 

If  there  exists  a node  that  performs  1>3[  O'.ol)] , then  a REQGnl) 
arrived  or  will  arrive  at  the  SINK  in  finite  time. 


Proof 

Let  PC y j * 1,  2,  ...  denotes  the  j-th  occurence  of  PCGnl). 

Given  a node  i and  a time  t such  that  Ti^2[i,  Of*. mil]  has  happened  before 
t,  if  PCj  is  the  last  PCQnll  before  t,  afterwhich  Ttj>2[i,  C^.ml)]  happened, 
then  define  E^  Ct)  ■ j • 

Property 

Given  a time  t,  suppose  that  k e SON.  Ct)  and  niCt)  ■ r.ktt)  ■ ml, 
then  EiCt)  S \Ct). 

Proof  of  the  Property 

Let  tl  be  the  time,  node  k was  last  set  to  be  a son  of  node  i before 
time  t.  This  can  be  done  only  via  T21[i]  or  T32[i].  Let  PC^  be  the  last 
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r 

proper  completion  before  tl.  By  Theorem  3.3  s^[PC^.]  i S2[ml]  , therefore 
Tv2  [i , QP  ,ml)  ] happened  after  PC ^ and  it  cannot  happen  again  before  t 
because  of  Theorem  3.2  iii). 

Hence  E^(tl)  * j 

The  occurence  of  T21  [i]  or  T32[i]  implies  that  a message  MSG(d**«)  has  been 
received  at  i from  k after  PCj  . By  Theorem  A.l  b)  this  message  was 
sent  by  k after  PCj  and  this  can  only  be  done  if  k performed  Tif/2[k]  after 
PCj . Since  E^  is  a nondecreasing  number,  then  Ek(tl)  i j.  Since  at 
time  t,  k still  belongs  to  SON^t),  then  node  i cannot  enter  state  S2 
on  the  interval  (tl,t)  unless  node  k has  entered  S2,  implying  that 
E^t)  s EkCt). 

q.e.d.  (the  property). 

We  may  now  continue  the  proof: 

By  Lemma  B.4  we  have  to  prove  this  Lemma  only  for  the  case  in  which  for  all 
nodes  in  the  network  we  have  n.  ( nl.  Therefore,  a node  that  effects 
T^3[  (i|/,ml)  ] cannot  effect  any  more  transitions  (by  property  R3) . Since 
the  number  of  links  in  the  network  is  finite,  then  only  a finite  number 
of  transitions  T<j/3[  (t|i,ml)]  can  be  executed.  If  T^3[  (i(;, ml)]  happens,  then 
there  exists  a node  which  detects  a failure  on  its  link  to  its  only  son 
and  executed  T^3[(ml,ml)]  (see  steps  B.2.1  - B.2.3,  B.2.7,  B.7.3,  B.10.3 
in  Section  3.3.3).  Define  B1  as  the  set  of  nodes  for  which  T^3[(ml,ml)] 
happens,  namely 


B1  ■ { i | T<|»3[t,,i, (ml, ml)]  happens). 
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Define 

B 

CASE  1 


CASE  2 


B2  as  a subset  of  nodes  of  B1  with  the  highest  Ej , namely 
! ■ {A  | l e B1  and  E,(t»)  * max  E.  (.t.).}. 

A **  . _ , 1 1 

leBl 


Suppose  there  exists  a node  l e B2  that  effects 

T23[tt.*,0nl,ml)]  or  T23[tlti.  Qnl.ml]]  (B- 12) 

Let  max  E.(t.l  * j.  Then  at  PC,  (by  Theorem  3.3) 
ieB2  1 3 

siCPCjl  i S2[ml].  Thus  the  first  node  i e B2  that  effects 

CB.12)  has  at  least  one  route  to  SINK  at  t^  (by  Theorem  3.1  i)). 

From  all  nodes  £ e B2  that  effects  (B.12)  while  having 

a route  to  SINK  at  t^,  let  us  choose  a node  qQ  such  that 

q1  e SONq  (tq  ) and  q^  t B2.  (£here  must  exist  such  a node 

since  SINK  i B2) ..  Because  of  the  property  proved  above  q1  t Bl, 

so  q1  does  not  enter  S3.  Also  by  Theorem  3.1,  s (t  1 «*  S2[ml] 

or  S2[ml]  and  q^  can  only  effect  T22  or.C2,  because  we  have 

showi  it  cannot  effect  Tf3,  and  it  cannot  effect  also  T21  unless 

it  receives  a message  MSGCd^**)  from  qQ,  which  cannot  be 

sent  since  q does  not  effect  T21. 
o 

Hence,  q.  will  detect  a failure  at  link  (q  ,q.)  and 

1 O A 

send  REQ(ml)  while  s^  ■ S2[ml]  or  S2[ml] , and  by  Lemma  B.S 
the  assertion  follows. 

Suppose  there  existsno  node  1 c B2  that  effects  (B.12),  i.e. 

every  node  l e B2  effects  T13[t, ,1, (ml ,ml)] . Let  q e B2 

denotes  a node  such  that 

d Ct  1 - min  (d.  (.t.-) } 

4-  1 A 


'0 


icB2 


When  q entered  SI [ml]  at  the  last  time  before  t , it  had 

0 Q 

its  preferred  son  p (see  step  B.4.14  in  Section  4.3.2.) 

% 


(B.  13) 


cannot  effect  T23  because  this  will  violate  CASE 


2, and  cannot  effect 


T13  because  this  violates  either  (B. 1 3 J or  CASE 
a failure  on  link  (p  ,qQ)  and  sends  REQCml). 


2.  Therefore  p 

q 

^o 


detects 


If  at  any  time  this  REQ(ml)  is  proct  ssed  by  a node  at  state  S2 
or  state  S2,  then  the  assertion  follows  by  Lemma  B.5.  Otherwise  the 
REQ(ml)  keeps  moving  through  nodes  at  SI [ml]  since  it  cannot  be  received 
by  a node  at  state  S3  because  this  violates  either  CASE  2 or  (B.13). 

The  REQCml)  is  forwarded  from  each  node  to  its  preferred  son,  thus  it 
moves  through  nodes  having  decreasing  d's  by  Theorem  4.1.  Even  if  the 
REQ(ml)  arrives  at  a node,  £ say,  with  p^  NIL,  then  node  l detected 
a failure  on  link  (Jl.p^)  and  p^  has  sent  REQCml)  and  by  Theorem  4.1 

d •'  d.  when  it  has  been  sent. 

Pi  i 


Since  for  all  nodes  i,  d.  >,  0,  di  is  an  integral  number  and  the 
only  node  with  d.  ■ 0 is  the  SINK,  the  REQCml}  will  arrive  at  SINK 
after  a finite  number  of  steps,  q.e.d. 


Lemma  B. 7 

If  a node  i sends  a REQOnll  while  s^  ■ SI [ml]  then  a REQCml]  arrived 
or  will  arrive  at  SINK  in  finite  time. 


Proof 

If  there  exists  a node  l such  that  s^  ■ S3 [ml]  then  q.e.d.  by 
Lemma  B.6.  Hence  we  may  assume  that  for  all  nodes  l in  the  network 
t S3[ml].  Also,  by  Lemma  B.4  we  know  that  the  REQCml)  sent  by  i may 
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A 


encounter  only  nodes  having  n^  = ml.  Thus  as  in  the  proof  of  Lemma  B.6 
the  REQCml)  either  arrives  at  a node  S2  or  S2  (q.e.d.  by  Lemma  B.5) 
or  moves  through  nodes  at  SI,  with  decreasing  d's  until  it  arrives  at 
SINK,  q.e.d. 


L 


J 
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APPENDIX  C 

In  this  appendix  we  give  the  program  that  simulates  the  operations 
done  by  individual  nodes . The  notations  in  the  program  are  the  same 
as  in  Section  3.3.3  except  that  S4  is  used  instead  of  S2.  Since 
only  an  individual  node  is  considered,  the  steps  of  determining 
the  routing  and  of  additions  of  links  are  not  essential  in  this 
simulation.  The  program  simulates  all  posssible  situations  in  a node's 
memory  location  and  all  the  messages  it  can  receive  in  each 
situation.  Apriori  knowledge  allows  us  not  to  check  several  situations. 

• 

As  a result,  this  program  shows  that  the  Finite-State-Machine  acts 
as  is  expected,  and  proves  property  R7  in  Appendix  A. 


4-  . 


i»l L F I ICNS  ll>M  K ) i 
N»Ji/«  NLMfcEF  OF  NEIGRECHS  */ 

MR  : de(ils  ; 

/*  UfctLAFAT  1C  N •/ 

OCL  ( J.RCUMlfclN  FIXED(2C,C); 

ill  mg  cfaria)  var riNC  ; 

Dll  FIIM  CRAF(S)  V«nYUGi 
uCL  FII(n)  chh<si  varying; 

LCL  ( h 1 (N)  , R n (N  ) ) ChAF  < 3 » ; 

UCL  (MK(N)iM(NI  ■OIKIMiCMIM  . 
L_lK(N),M,D,CT.MXl.MXl.S.Sl.PI,Pl,Nl,  Dl,  01, 
011  ,012)  0 IN  F I XE  o; 

J • C » 

/*  CONSTANTS  «/ 

NtUNU»=4i  /*  SCCc  C CU  NT  E R FUMEEF  »/ 

InFsSGC;  /•  1HIS  IS  CCNSICEHEC  AS  INFINITE  */ 

C.lMlJscj;  /«  MARGINAL  CE  LA  Y OF  LINK  (1,1)  */ 

0_lK(2)=lj;  /*  MARGINAL  CELAV  CF  LINK  (1,2)  */ 

J_lK.U)  = lSi  /*  MARGINAL  DELAY  OF  LINK  (1,3)  */ 


/«* 


IRC  N JOE  * S MEMORY  LCCATICNS  ( 
F I l ( 1)  = 'OF  • ; /*  LINK  (1,1)  IS 
F II  (Z)s'uF'  i /*  LINK  (1,2)  IS 
Fli(J)=*LF*;  /*  LINK  (1,3)  15 

lM(1)  = 15;  /«  LAST  C RECEIVED 
LKl(2)=lCi  /•  u AS  T C RECEIVEC 
OK  1 ( 3) =S I /*  LAST  C RECEIVFC 


INITIALLY)  **/ 
OFERATICNAL  */ 
1PERATICNAL  */ 
CFEFATICNAL  */ 
CN  LINK  (1,1) 
CN  LINK  (1,2) 

C N LINK  (1,3) 


*/ 

*/ 

*/ 


•til  (J)s'ml'  ; 


f l 5 : m x i = a ; 


m x : M 1 1 ) =o  ; 

n u : n i ( 2 ) = o ; 

M2  : ni  (3  ) = o ; 

n lo  : s l = l ; 

sTa:oi=io; 

UELiKl  1 (1)  **nil*  s 
jCn  i:ki  i(2)s  'ml  • ; 

/*  THE  MESSAGE  */ 

Mo  = 'FAIL'; 
jL N2 1 M= 2 i 

ngm:c-i o ; 

Ot : l » l ; 

n e i : * = „ ♦ l ; 

/*  AHhICRI  KNCALtOGE  ALLCA  LS  NCT  TC  CFECK  TFE  FOLLOWING  */ 

IF  hU  (1  ) *•  SC  N 1 A «U(<)='NIL'  TFEN  GOTO  S02CHA;/*»E  RAVE 
lrECKEO  THIS  SITUATION  */ 

IF  mxIKNCONOM  Tt-£N  CO  TC  MXCFA;  / »Mx  I IS  NEVEW  LESS  THAN  Ni*/ 
IF  Ml.=  *MS«»  4 HI  1 ( L)  * *SC  N ' 4 M<  N C ON  L M THEN  GOTO  END1;/*IMP0- 

SolcLE  EY  LEMMA  A . 1 ,C  */ 

IF  l M X 1 <N1  ( 1)  | M X 1 <N  1 ( 2 ) | MXKNlt.1))  THEN  GOTO  N3CFA) 


jcf  cCn  = c ; 

DO  k = i ic  n ; 

IF  hlllKIs'SGN'  THEN  JCFECK=1; 

end; 

IF  al-=2  4 (JCFcCK=0  |Cl=INF  ) THEN  GOTO  SC2CHAJ 
IF  31=3  « ( J CF  5CK-,  = 0 |Cli=INF  ) TFEN  GOTO  SO  2CH  A ; 

1FIJCRECK  = 0 a Ul-,  = INF)|  ( jCF  f 4 Cl  = lNF)  TFEN  GOTO  S02CHA; 


IF  3 1 —3  TFEN  CO!  / *T  hE  NCCE  CANT  EE  IN  S3  IF  THE  FOLLOWING  HAPPENS*/ 

j4  = L ; 

cl  k= i ton; 

IF  FII(K)S*LF*  4 MX1=N1(K)  4 MXl>NCCNOM  4 OKI  (KJ-.S1NF  THEN  J4=l  i 

end  ; 

IF  jA  = l TFEN  C OT  0 ENC1  ; 
fc.NO ; 

IF  31=2  TFEN  CJ{/*TF5  NCCE  CANT  EE  IN  52  IF  THE  FOLLOWING  HAPPENS'  / 
ji  = g;  j£  = l;  — 3 = c ; 
ol  k= i ic  n ; 

1 F FI1(K)='LP*  4 N1(K.)-*=NCCNUN  TFEN  Jl=l{ 

If  FllUJs'LF'  4 OK1IMOC1  T*-EN  -2  = 15 
ir  K 1 l (K  )=  • SCN  ' 4 OK  1(K  )=  I NF  ThEN  -2=11 

r.  NO  1 

IF  J 1 = C 4 -2*1  4 J 3=  C TFEN  GOTO  ENCli 

enc  ; 


IF  SI=l  THEN  CC;/»  the  FCDE  CAN'T  3E  IN  STATE  l ■/ 
j2  = 0 ; 

DC  k.  = 1 TC  A • 

IF  KlllKIs'SCN'  THEN  IF  (NJIKI-jOM  | OKI(K):lNTI  THEN  J2  = IJ 

end  ; 

IF  Jc-Q.  TFEN  GCTC  EN  C 1 ; 

END  i 

IF  al  =c  I Sl=4  T F EN  cc;  /•  TfE  NOCE  CAN'T  BE  IN  THESE  STATES  */ 
J2  * C ; 

cc  n=  i tc  n ; 

IF  Mil  (K)s'ECN'  T FE  N IF  (N1(K)-.  = MX1  I CK1(K)=INFI  TFiEN  J2=I  ; 

trio ; 

IF  J2  = C A maianjdnum  tfen  ggtu  enoi; 

cNO ; 


DC  *=1  TC  n; 

NIK(  K1  = N1  <K)  ; kllK)sFIl(K);  FI  («)>FI1  (K  ) i D I K ( K I a OK I ( K ) J 

end  ; 

MISKXIJ  S = S1  ; CI=0l  ;M=NCCNUW; 

/*  TFE  NCDE  STARTS  TO  FPCCESS  TFE  MESSAGE  */ 

IF  Mu*  'FAIL*  TFEN  DO ; 

FI  lt)  = 'CC*N'  ; 

c t — w ; 

uOTO  FSM  ; 

E no  ; 

I F MG  = • Mats  • TFEN  CO  ; 

IF  F I ( L > = ' RE  AOY  • TFEN  FKUs'UF'S 
MXlL  >*W  ; 

UMU»CiC_IF<L  ) ; 

IF  C=INF  THEN  UIK(L)=IFF; 

MA  l = MA* ( M I M A I ) ; 

CT  =o  ; 

aC  T C F S M ; 

END  i 


/**»*•***  FINITE  STATE  MACHINE  *«*****«/ 

«(  »««**<*/ 
/.»******  TRANSITION  12  •*****♦*/ 

/4M*M*****4**M4*M»M*t**«M**»M*»*»«***/ 

F a M ! 


IF  S=  1 A C T *0  THEN  CC! 

j l = J • a 2 = c ; 
cc  a — i re  n ; 

IF  f,l  Uts'SCN'  TFEN  IF  FKKHs'UF'  TFEN  Jl  = lJ 
IF  miK)s'SCN'  TFEN  IF  (Mk(K)->:M>I  | D 1 K ( K ) = I NF  I 
TFEN  .2=1; 

end  ; 

IF  J1=C  A j2=  0 THEN  DO; 

IF  MG='MSG‘  A M<N  I TFEN  POT  L I S T I • C C N TR  A C I CT  ION  TCI  FACT  I 2 • ) SK  I P ; 
EL  Sc  0 c ; 

/*PUT  L 1ST!  ‘TRANSIT ICN  12  OCCUREC  * ISKIP  ;*/ 

goto  opcate; 

End  ; 

gctc  e n c i ; 
end; 


end  ; 

/*»*»4******»«*«444 4 44 4*444 444** 4 44****** ******* 4>  ***********  / 

/**«**«»*  TRANSIT  ICN  TC  STATE  J **«***•*/ 

/••44*«4**44****4*444444*  4 4 • 4 4 4 4 44 44* 44444*4*444444444444444/ 


if  is=i  | s=2  | s=a>  a ct  = o tfen  co ; 
a a a * c ; 

CC  K=  1 TC  N ; 

IF  A = L TFEN  GOT C X3  ; 

IF  Hi (K)s'SCN*  THEN  JA3=l; 

as: end; 

IF  (aX3=J  A H(lJs'SCF')  A <(NG*‘M5G‘  A C=  INF  1 1 ( MO* • FA IL ‘ ) ) 
THEN  DC  ; 

IF  MGs’MSC*  A M<N I TFEN  CC; 

IF  S*1  TFEN  POT  LI S T ( 'CC NTRAOIC T I C N TC  FACT  IJ'JSKIP; 

IF  S =2  TFEN  PUT  L I S T ( • CC N TW AC  I C T 1C N TO  FACT  2J')SKlP; 

IF  E =4  • The  N PuT  LIST!'  CCNTRAC  ICT  ICN  TO  FACT  A?  ISKlPi 

vjjto  End  i ; 

l n o ; • 

/•IF  S=I  THEN  PUT  L I 3T ( • TP ANS  I T ICN  13  C C CURE 0 • . J ) SK I P ; •/ 

/•IF  a*  2 TFEN  HOT  L I S T ( • TR A N S I T I C N 23  CCC L WE D • . J I SKI P ; • / 

/•IF  S=A  TFEN  HUT  LIST  ('TRANSIT  I ON  * 3 C CCURE D • . J ) SK I P I •/ 


1 


C H A N 3 s l 


*/ 

*/ 

*/ 


r 


/«•»».*»» 
/•*«***• 
/**«•***  4 

t hc  n o i ; 


i 0=1 NF> I (MG=«FAIL‘ ) ) 


IF  S = l 4 C T = 
jC i =0  ; 

oc  k = i tc  n; 

IF  K = C THEN  GOTO  JCH ; 

if  kiikj='so*  r hen  j c i = i ; 

jCH  • - NO  t 

IF  UCU1  a l-[(L)='SCN'l  4 ui»5s"4«6' 

THEN  C C • 

rIIlm'Ml1  ; 

/ iPJT  L laM  'ChANGE  1 CC  C LRE.  0 * • 0 ) SKIP  J*/  GCTU  FSM  ; 

EN3! 
t NO  • 

/44****4»****»*44444*4****»***********A4^**t*/ 
/*»•*»»**  1 R A N S 1 T 1 C N 21  ♦*•*****/ 

IF  ->=2  TFEN  CC  i 
jl  =0  i -2  = 0 ; . 3 =0  1 

DO  K.  = 1 T J N ; 

l F l-ilUls'lP1  4 NIMK1-.SM  THEN  J 1 - 1 1 
IF  FI(»l«'Uf'  4 D I K ( * ) < =0  I THEN  J2  = i; 

IF  K i (K  ) : 'iCN  ' 4 OIMOSINF  THEN  -3  = 1*, 

:ND; 

if  M=l»J<I  4 .1=0  4 J2  = l 4 .3=0  TFEN  CC  J 
IF  C T = C THEN  IF  MG->  = • * St  • THFN  GCTC  a; 

Ir  0 1=  INF  Tl-EN  Cj; 

PLT  Liil(  'CCNlkAOICIICN  TC  FACT  21*  ) S K I P ; 

vjoto  en  d i ; 
cno; 

tLit  cc; 

011=  10000 ; 
oc  K = 1 tc  f ; 

IF  FllH)  = ' LP  ' THE  N c c ; 

C 12  = M IMCItiOIK(K)); 

MK(K)«Ci 

IF  0 1 2 <L  l 1 THEN  P 1 =K  : 

C I 1 =0  12  ; 

eno  ; 

=n  □ ; 

Hi  (F i ) = • scf*  ; 
ct  = l ; 

= = i ; 

/*PoT  LIST!  ' ThANil  TICK  21  CC C UPE C • > S K l P ; * / 

GU  T 0 F5M  ; 

Aii.  no  ; 

CrlO  i 
t NO  • 


TfiANSITICN  22 
j S = A ) 4 m[>Nl  THEN  cc; 


0 * 


A 2 


/» *4**44* 

/'*  **«•****»■ 

IF  U S = 2 4 C T = 0 ) 

Jl  = 0 • j2  = 0 « 

LC  K = l T C N ; 

IF  kllKjs'iCN'  THEN  IF  FI  (K)-.s'lP*  THEN  Jl=i; 

IF  h 1 (m  = *5  tN'  THEN  IF  (MK(K)-sMXl  | DlK(K)sINF) 
THEN  .2  = 1; 

lnj  : 

IF  J1=0  4 -2=0  THEN  C3; 

/*  I F S=  2 TFEN  POT  LIST  < • TPanS  I T ICf>  22  CC CUREC • ) SK I P ; »/ 
/‘IF  S=A  THEN  PUT  L I S T ( *TF AN s I T I CN  *2  CCC L RE D • > S< I P J * / 
GuTC  LFCATt; 

Enu  ; 

en  o ; 

/»**4**********t***«***«*4444**44* 


««**«***/ 

*»••**«*/ 


2 A *****•*•/ 


/***•*»«*  T R A N c l T l C N 

/**444********<44<>«*»**44l*4t4*l 

IF  2 = 2 i C T = C THcN  CC! 

JO  l = u i 

Co  K = 1 T L X 1 

IF  n=c  thef  gctc  ach; 
if  kkms'scn1  then  jci=i; 
a Ch : c no ; 

IF  ( ( jC  1 = 1 4 A I ( L J = * S C F ' ) 4 ((AG*‘»SC‘  4 Cx  INF)  | ( MG=  • FA  ll_  ' >1  > 


l -lUL  )is‘3JN*  4 MG=«FA[L‘)  the  0 DC! 
mius'ML'  ; 
c f = 1 i 
u = a ; 

/«FOT  L 1 S r ( • T F ANb l T I C N 24  CCCJP5C*  ) S A I P • 
yil  TlJ  F H * J 

enc  ; 

i no  ; 


*/ 


I 


144 


/»*<«  44  *144  44***4444****** *»«***/ 
/**4**»4  CHANCE  4 4**»****/ 
/•«**  4*  4*** »****«  ***************/ 


1 F j=4  4 _ T = v>  THciN  DJI 

■jC  i - o ; 

UO  • I To  \ i 
IF  N = L T rEN  CCTC  JXX  ; 

if  «i<k)s*scm  then  jci=i; 

j xx : end  ; 

IF  CoCl=l  A FllL>=*SCN*)  4 (IK>"»SC  4 0*  l NlF  ) | ( MG  = • F A IL  * ) > 

iMti'i  oc; 

H I tL  > = *n  xu  • ; 

/*HuT  L l sT  I • CF ANGfc  4 C C CU  N = C • . J » i K I F J */  GCTO  F SM  i 


En  u ; 

ENC  ; 

/4**«4**M*MMI***M»4<*4*«MM******M*****/ 
/**44*«4*  T h A N S l T l 3 N 3 2 ********/ 
/***««* .44A444444444444444444444*************/ 


iF  i=J  S M>l>M  T HE  N CCJ 

0*4  -0  « j£  =J  I 

dl  k=i  to  n; 

IF  FMMs'uF'  4 MXIsMMK)  4 C l « < K )-.=  I NF 
THEN  .«:l  I 

IF  k I ( K > =•  « C F • TF£F  -£*i; 

SN  C I 

IF  J**  = 1 THEN  CCJ 

IF  Jt-=0  | Cl->  = INF  IF  EF  cc  ; 

PUT  L IoT(  • CGNTKAOI  CT  ICN  TO  FACT  E£')SKIP; 
vi  J TC  6 NO  1 • 

— no  ; 

c 1 1 = icooo  ; 
jc  * * 1 tc  n ; 

IF  F I ( K ) * ' ' JP  * 4 NIK(K)S»>I  TF  EN  CCI 
C I 2=M  |M  C I 1 . 0 IK(  K J ) ; 

IF  C I 2<C  I l THEN  F I = K i 

ci  1 = 0 u ; 
d n c ; 
end  ; 

k 1 1 p i )=  • scn  • ; 
m = *x  i ; 

\j  i sy  i k i f*  1 1 ; 

= = <i ; 

c t = i ; 

/ *P  o T L 1 STI • TFANSI T IC F ~~2  CCC  UPc  C • ) « K l F ”,  « / 
oJTCJ  F>iM  ; 
lnu ; 
t wo  ; 

oCTC  E.NC2; 

upc  a r c i 

t i =i ccjc  ; 
o c a = i to  n ; 

IF  MlKls'UP'  J MMH)S4XI  THEM  0 I =M  I N ( L)  l iD  I Kl  N M J 
i.  no; 
n »***  * i ; 
c t = i ; 

cofj  pjm  ; 

FmIu:  c l =i  nf  ; 

IF  Mus  'M‘G'  then  fi=*>; 

FllLls'ML*; 

ccr.  Fjx  ; 

/**•*«***  r F t C K I N G *44*4***/ 

LNC2  : 

ir  i - J ThcN  C C • 

Oo  n = I TC  N • 

lr  FI(<Jr#CF*  4 NlK  (X  )*4XI  4 HUM  4 0IK(K)-«  = INF  THEN  DUi 

plT  LI  i T I •£  fi  H C 0 

The  M.CE  FIST  H«vc  LEFT  Sl'.jJSKlFJ 

i no  ; 


ct  * = i ra  n; 

IF  4 MK<k)-*M  TFEN  jlsi; 

IF  FI(*)*'UP'  4 DIK(K)<=CI  TFEN  J2=i; 

IF  hi  (K)  * • S C N • 4 CIMMMNF  TFEN  J2  = l; 

tNO  ; 

IF  M=M1  4 Jlsj  4 J 2 = 1 i J 3=  C TFEN  CO; 

PL  T LlSIl'E  P P C R 

T He  NO  C 6 ML  S I FAVt  L£F1  S2,.J)$KIP; 

CNO ; 

z no  ; 

IF  3=4  4 XXl>M  TFEN  CC; 

oc  k=  i ro  n ; 

IF  hlUM'iCN'  TFEN  IF  FKKI-is'UP'  THEN  Jl=i; 

IF  KKKjs'SLN*  THEN  IF  (MMK)-.s»Xl  | CIK(K)slNF) 
T F eN  J2  = I ; 

end  ; 

IF  Jl  = c 4 j2  = C THEN  CC; 

PUT  L I S T ( • E P P C P 

Tnc  NCCE  MLSI  HAVE  LEFT  S4*,J)$KlP; 

enj; 

iNO ; 


z nj i ; » 

IF  L=J  TFEN  GCTC  VESCHA; 
t*L4i  ; 
wCTC  n — i ; 

MEoCFA:  IF  MG=‘F3G*  TFEN  GC  TO  TCFX  ! 

pijs*  xSj1  ; 
vjJTC  DE  ; 

LChhI  IF  J-l  NF  THEN  CC.TC  FCFAJ 
Q = lNF  ; 

LCTO  CE  ; 

MLHA ; IF  M=E  THtN  GCTC  SC2CHA; 
h = f ♦ 2 ; 

GCTC  N L N • 


iC^CM:  IF  Nil  (2)-*  SC  N • TFEN  GCTC  50ICPA; 
Willi  I* 'SiN*  i 
CCTC  SC  N 2 ; 

JUILHAI  IF  hIHn*'S3h'  TFEN  GCTC  cccfa; 

p i i 1 1 i=  *scn • ; 

• CUTC  SCM  ; 

cccha:  if  ci=inf  then  gctc  scfa; 

C 1 = 1 NF  ; 

GC  TO  CcL  ; 

mm:  if  $1=4  then  gctc  n3Cfa; 

s i*s  if  i ; 
gctc  sta; 

Olha  : IF  NKOOMXl  TFEN  G'JTC  NSCfa; 

IF  N 1 (2  ) =e  TFEN  C-CTC  N 2 C F a ; 

IF  M ( 3 ) =4  THEN  M (3  > =6  J 
IF  N 1 ( 3 ) = 0 TFEN  N K 3 ) =4  ; 

GCTC  M3; 

n2(_ha:  IF  N1(2)>MXI  TFEN  GC  TO  NICFA; 

IF  Nl(2)=0  TFEN  CCTO  NlCFM 
IF  Nl( 2) =4  THEF  m (2) *6; 

IF  N1(2I  = 0 TF  = N N 1 ( 2 ) =4  ; 

CCTC  M2; 


! 


gigfa;  if  M(1)>MX1  TFEN  CCTC  *xcfa; 
IF  N1  ( 1 I =o  THEN  CCTC  NXCF«; 

IF  N H 1 ) = 4 TFEN  N t ( 1 ) =G ; 

IF  N1  ( I »=G  TFEN  Mil  ) = 4 ; 

GCTC  mi; 

m.i-a:  IF  vxl=o  ThtN  GCTC  f ’CFAi 

n > i »c  ; 

GC  T C M x ; 

f 3«»ha  : 

» n umk  : E n o vp  ; 

C.NJ  MrJS  F ; 
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